{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,16]],"date-time":"2026-09-16T16:33:04Z","timestamp":1789576384229,"version":"build-2803163510"},"reference-count":0,"publisher":"Korea Health Personnel Licensing Examination Institute","content-domain":{"domain":["jeehp.org"],"crossmark-restriction":true},"short-container-title":["J Educ Eval Health Prof"],"abstract":"<jats:p>Purpose: This study systematically evaluates and compares the performance of large language models (LLMs) in answering medical licensing examination questions. By conducting subgroup analyses based on language, question format, and model type, this meta-analysis aims to provide a comprehensive overview of LLM capabilities in medical education and clinical decision-making.Methods: This systematic review, registered in PROSPERO and following PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines, searched MEDLINE (PubMed), Scopus, and Web of Science for relevant articles published up to February 1, 2025. The search strategy included Medical Subject Headings (MeSH) terms and keywords related to (\u201cChatGPT\u201d OR \u201cGPT\u201d OR \u201cLLM variants\u201d) AND (\u201cmedical licensing exam*\u201d OR \u201cmedical exam*\u201d OR \u201cmedical education\u201d OR \u201cradiology exam*\u201d). Eligible studies evaluated LLM accuracy on medical licensing examination questions. Pooled accuracy was estimated using a random-effects model, with subgroup analyses by LLM type, language, and question format. Publication bias was assessed using Egger\u2019s regression test.Results: This systematic review identified 2,404 studies. After removing duplicates and excluding irrelevant articles through title and abstract screening, 36 studies were included after full-text review. The pooled accuracy was 72% (95% confidence interval, 70.0% to 75.0%) with high heterogeneity (I2=99%, P&lt;0.001). Among LLMs, GPT-4 achieved the highest accuracy (81%), followed by Bing (79%), Claude (74%), Gemini\/Bard (70%), and GPT-3.5 (60%) (P=0.001). Performance differences across languages (range, 62% in Polish to 77% in German) were not statistically significant (P=0.170).Conclusion: LLMs, particularly GPT-4, can match or exceed medical students\u2019 examination performance and may serve as supportive educational tools. However, due to variability and the risk of errors, they should be used cautiously as complements rather than replacements for traditional learning methods.<\/jats:p>","DOI":"10.3352\/jeehp.2025.22.36","type":"journal-article","created":{"date-parts":[[2025,11,17]],"date-time":"2025-11-17T22:50:53Z","timestamp":1763419853000},"page":"36","update-policy":"https:\/\/doi.org\/10.3352\/crossmark_policy","source":"Crossref","is-referenced-by-count":12,"title":["Performance of large language models in medical licensing examinations: a systematic review and meta-analysis"],"prefix":"10.3352","volume":"22","author":[{"ORCID":"https:\/\/orcid.org\/0009-0009-6099-5381","authenticated-orcid":true,"given":"Haniyeh","family":"Nouri","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4591-2209","authenticated-orcid":true,"given":"Abdollah","family":"Mahdavi","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4289-4941","authenticated-orcid":true,"given":"Ali","family":"Abedi","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3013-6035","authenticated-orcid":true,"given":"Alireza","family":"Mohammadnia","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0008-2325-9003","authenticated-orcid":true,"given":"Mahnaz","family":"Hamedan","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7324-6050","authenticated-orcid":true,"given":"Masoud","family":"Amanzadeh","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"14441","published-online":{"date-parts":[[2025,11,18]]},"container-title":["Journal of Educational Evaluation for Health Professions"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/jeehp.org\/upload\/pdf\/jeehp-22-36.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/jeehp.org\/DOIx.php?id=10.3352\/jeehp.2025.22.36","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/jeehp.org\/upload\/pdf\/jeehp-22-36.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,11,17]],"date-time":"2025-11-17T22:50:53Z","timestamp":1763419853000},"score":1,"resource":{"primary":{"URL":"http:\/\/jeehp.org\/DOIx.php?id=10.3352\/jeehp.2025.22.36"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,11,18]]},"references-count":0,"URL":"https:\/\/doi.org\/10.3352\/jeehp.2025.22.36","relation":{},"ISSN":["1975-5937"],"issn-type":[{"value":"1975-5937","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,11,18]]},"assertion":[{"value":"2025-09-23","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-10-30","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-11-18","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}