{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,20]],"date-time":"2026-08-20T14:56:33Z","timestamp":1787237793829,"version":"build-2736575974"},"reference-count":25,"publisher":"PAGEPress Publications","issue":"2","license":[{"start":{"date-parts":[[2024,11,28]],"date-time":"2024-11-28T00:00:00Z","timestamp":1732752000000},"content-version":"unspecified","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by-nc\/4.0"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Dermatol Reports"],"abstract":"<jats:p>The large language model (LLM) ChatGPT can answer open-ended and complex questions, but its accuracy in providing reliable medical information requires a careful assessment. As part of the AI-CHECK (Artificial Intelligence for CME Health E-learning Contents and Knowledge) study, aimed at evaluating the potential of ChatGPT in continuous medical education (CME), we compared ChatGPT-generated educational content to the recommendations of the National Institute for Health and Care Excellence (NICE) guidelines on acne vulgaris. ChatGPT version 4 was exposed to a 23-item questionnaire developed by an experienced dermatologist. A panel of five dermatologists rated the answers positively in terms of \u201cquality\u201d (87.8%), \u201creadability\u201d (94.8%), \u201caccuracy\u201d (75.7%), \u201cthoroughness\u201d (85.2%), and \u201cconsistency\u201d with guidelines (76.8%). The references provided by ChatGPT obtained positive ratings for \u201cpertinence\u201d (94.6%), \u201crelevance\u201d (91.2%), and \u201cupdate\u201d (62.3%). The internal reproducibility was adequate both for answers (93.5%) and references (67.4%). Answers related to issues of uncertainty and\/or controversy in the scientific community scored the lowest. This study underscores the need to develop rigorous evaluation criteria for AI-generated medical content and for expert oversight to ensure accuracy and guideline adherence.<\/jats:p>","DOI":"10.4081\/dr.2024.10138","type":"journal-article","created":{"date-parts":[[2024,11,28]],"date-time":"2024-11-28T03:02:13Z","timestamp":1732762933000},"source":"Crossref","is-referenced-by-count":5,"title":["Application of ChatGPT as a content generation tool in continuing medical education: acne as a test topic"],"prefix":"10.4081","volume":"17","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-3160-2835","authenticated-orcid":false,"given":"Luigi","family":"Naldi","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Vincenzo","family":"Bettoli","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5719-9896","authenticated-orcid":false,"given":"Eugenio","family":"Santoro","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5195-5208","authenticated-orcid":false,"given":"Maria Rosa","family":"Valetto","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0009-1626-0855","authenticated-orcid":false,"given":"Anna","family":"Bolzon","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5014-1122","authenticated-orcid":false,"given":"Fortunato","family":"Cassalia","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8161-6138","authenticated-orcid":false,"given":"Simone","family":"Cazzaniga","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7570-7639","authenticated-orcid":false,"given":"Sergio","family":"Cima","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0003-8656-2927","authenticated-orcid":false,"given":"Andrea","family":"Danese","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Silvia","family":"Emendi","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0008-2496-7255","authenticated-orcid":false,"given":"Monica","family":"Ponzano","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0003-8120-6923","authenticated-orcid":false,"given":"Nicoletta","family":"Scarpa","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3565-4787","authenticated-orcid":false,"given":"Pietro","family":"Dri","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"2549","published-online":{"date-parts":[[2024,11,28]]},"reference":[{"key":"28175","unstructured":"1. OpenAI. ChatGPT. https:\/\/chat.openai.com\/chat"},{"key":"28176","doi-asserted-by":"crossref","unstructured":"2. Noy S, Zhang W. Experimental evidence on the productivity effects of generative artificial intelligence. Science 2023;381:187-92.","DOI":"10.1126\/science.adh2586"},{"key":"28177","doi-asserted-by":"crossref","unstructured":"3. Mesk\u00f3 B, Topol EJ. The imperative for regulatory oversight of large language models (or generative AI) in healthcare. NPJ Digit Med 2023;6:120.","DOI":"10.1038\/s41746-023-00873-0"},{"key":"28178","doi-asserted-by":"crossref","unstructured":"4. Dave T, Athaluri SA, Singh S. ChatGPT in medicine: an overview of its applications, advantages, limitations, future prospects, and ethical considerations. Front Artif Intell 2023;6:1169595.","DOI":"10.3389\/frai.2023.1169595"},{"key":"28179","doi-asserted-by":"crossref","unstructured":"5. Safranek CW, Sidamon-Eristoff AE, Gilson A, Chartash D. The role of large language models in medical education: applications and implications. JMIR Med Educ 2023;9:e50945.","DOI":"10.2196\/50945"},{"key":"28180","doi-asserted-by":"crossref","unstructured":"6. Eysenbach G. The role of ChatGPT, Generative Language Models, and Artificial Intelligence in medical education: a conversation with ChatGPT and a call for papers. JMIR Med Educ 2023;9:e46885.","DOI":"10.2196\/46885"},{"key":"28181","doi-asserted-by":"crossref","unstructured":"7. Sallam M. ChatGPT utility in healthcare education, research, and practice: systematic review on the promising perspectives and valid concerns. Healthcare (Basel) 2023;11:887.","DOI":"10.3390\/healthcare11060887"},{"key":"28182","doi-asserted-by":"crossref","unstructured":"8. Shah NH, Entwistle D, Pfeffer MA. Creation and adoption of Large Language Models in medicine. JAMA 2023;330:866-9.","DOI":"10.1001\/jama.2023.14217"},{"key":"28183","doi-asserted-by":"crossref","unstructured":"9. Gordon ER, Trager MH, Kontos D, et al. Ethical considerations for artificial intelligence in dermatology: a scoping review. Br J Dermatol 2024;190:789-97.","DOI":"10.1093\/bjd\/ljae040"},{"key":"28184","doi-asserted-by":"crossref","unstructured":"10. Chen S, Kann BH, Foote MB, et al. Use of Artificial Intelligence chatbots for cancer treatment information. JAMA Oncol 2023;9:1459-62.","DOI":"10.1001\/jamaoncol.2023.2954"},{"key":"28185","doi-asserted-by":"crossref","unstructured":"11. Ferreira AL, Chu B, Grant-Kels JM, et al. Evaluation of ChatGPT dermatology responses to common patient queries. JMIR Dermatol 2023;6:e49280.","DOI":"10.2196\/49280"},{"key":"28186","doi-asserted-by":"crossref","unstructured":"12. Goodman RS, Patrinely JR, Stone CA Jr, et al. Accuracy and reliability of chatbot responses to physician questions. JAMA Netw Open 2023;6:e2336483.","DOI":"10.1001\/jamanetworkopen.2023.36483"},{"key":"28187","doi-asserted-by":"crossref","unstructured":"13. Lam Hoai XL, Simonart T. Comparing meta-analyses with ChatGPT in the evaluation of the effectiveness and tolerance of systemic therapies in moderate-to-severe plaque psoriasis. J Clin Med 2023;12:5410.","DOI":"10.3390\/jcm12165410"},{"key":"28188","doi-asserted-by":"crossref","unstructured":"14. Rossettini G, Cook C, Palese A, et al. Pros and cons of using Artificial Intelligence chatbots for musculoskeletal rehabilitation management. J Orthop Sports Phys Ther 2023;53:1-7.","DOI":"10.2519\/jospt.2023.12000"},{"key":"28189","doi-asserted-by":"crossref","unstructured":"15. Temsah O, Khan SA, Chaiah Y, et al. Overview of early ChatGPT's presence in medical literature: insights from a hybrid literature review by ChatGPT and human experts. Cureus 2023;15:e37281.","DOI":"10.7759\/cureus.37281"},{"key":"28190","doi-asserted-by":"crossref","unstructured":"16. Bettoli V, Naldi L, Santoro E, et al. ChatGPT and acne: Accuracy and reliability of the information provided-The AI-check study. J Eur Acad Dermatol Venereol 2024.","DOI":"10.1111\/jdv.20324"},{"key":"28191","unstructured":"17. National Institute for Health and Care Excellence (NICE) Acne vulgaris: management. NICE guideline 198. Available from: https:\/\/www.nice.org.uk\/guidance\/ng198\/resources\/acne-vulgaris-management-pdf-66142088866501."},{"key":"28192","doi-asserted-by":"crossref","unstructured":"18. Nast A, Dr\u00e9no B, Bettoli V, et al. European evidence-based (S3) guideline for the treatment of acne - update 2016 - short version. J Eur Acad Dermatol Venereol 2016;30:1261-8.","DOI":"10.1111\/jdv.13776"},{"key":"28193","doi-asserted-by":"crossref","unstructured":"19. Reynolds RV, Yeung H, Cheng CE, et al. Guidelines of care for the management of acne vulgaris. J Am Acad Dermatol 2024;90:1006.e1-1006.e30.","DOI":"10.1016\/j.jaad.2023.12.017"},{"key":"28194","doi-asserted-by":"crossref","unstructured":"20. Zaenglein AL, Pathy AL, Schlosser BJ, et al. Guidelines of care for the management of acne vulgaris. J Am Acad Dermatol 2016;74:945-73.e33.","DOI":"10.1016\/j.jaad.2015.12.037"},{"key":"28195","doi-asserted-by":"crossref","unstructured":"21. Lakdawala N, Channa L, Gronbeck C, et al. Assessing the accuracy and comprehensiveness of ChatGPT in offering clinical guidance for atopic dermatitis and acne vulgaris. JMIR Dermatol 2023;6:e50409.","DOI":"10.2196\/50409"},{"key":"28196","doi-asserted-by":"crossref","unstructured":"22. Cirone K, Akrout M, Abid L, Oakley A. Assessing the utility of multimodal Large Language Models (GPT-4 Vision and Large Language and Vision Assistant) in identifying melanoma across different skin tones. JMIR Dermatol 2024;7:e55508.","DOI":"10.2196\/55508"},{"key":"28197","doi-asserted-by":"crossref","unstructured":"23. Reynolds K, Tejasvi T. Potential use of ChatGPT in responding to patient questions and creating patient resources. JMIR Dermatol 2024;7:e48451.","DOI":"10.2196\/48451"},{"key":"28198","doi-asserted-by":"crossref","unstructured":"24. O'Hagan R, Poplausky D, Young JN, et al. The accuracy and appropriateness of ChatGPT responses on nonmelanoma skin cancer information using zero-shot chain of thought prompting. JMIR Dermatol 2023;6:e49889.","DOI":"10.2196\/49889"},{"key":"28199","doi-asserted-by":"crossref","unstructured":"25. Charvet-Berard AI, Chopard P, Perneger TV. Measuring quality of patient information documents with an expanded EQIP scale. Patient Educ Couns 2008;70:407-11.","DOI":"10.1016\/j.pec.2007.11.018"}],"container-title":["Dermatology Reports"],"original-title":[],"link":[{"URL":"https:\/\/journals.pagepress.net\/dr\/article\/download\/10138\/9510","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.pagepress.net\/dr\/article\/download\/10138\/9693","content-type":"text\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.pagepress.net\/dr\/article\/download\/10138\/9509","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,3,20]],"date-time":"2026-03-20T10:30:22Z","timestamp":1774002622000},"score":1,"resource":{"primary":{"URL":"https:\/\/journals.pagepress.net\/dr\/article\/view\/10138"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,11,28]]},"references-count":25,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2025,5,23]]}},"URL":"https:\/\/doi.org\/10.4081\/dr.2024.10138","relation":{},"ISSN":["2036-7406","2036-7392"],"issn-type":[{"value":"2036-7406","type":"electronic"},{"value":"2036-7392","type":"print"}],"subject":[],"published":{"date-parts":[[2024,11,28]]}}}