{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,16]],"date-time":"2026-06-16T18:53:40Z","timestamp":1781636020604,"version":"3.54.5"},"reference-count":31,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2026,2,11]],"date-time":"2026-02-11T00:00:00Z","timestamp":1770768000000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2026,3,17]],"date-time":"2026-03-17T00:00:00Z","timestamp":1773705600000},"content-version":"vor","delay-in-days":34,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["BMC Med Educ"],"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:sec>\n                    <jats:title>Background<\/jats:title>\n                    <jats:p>Previous studies of ChatGPT performance in the field of medical exams have reached contradictory results. The performance of ChatGPT in languages other than English, including Arabic, which is the official language of medical education and practice in many countries, has yet to be explored. We aim to evaluate the performance of ChatGPT in Arabic-language Israeli OBGYN medical licensure exams for foreign university alumni.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Methods<\/jats:title>\n                    <jats:p>We conducted a performance study using a consecutive sample of text-based multiple-choice questions, originated from authentic Arabic-language Israeli OBGYN medical licensure exams for foreign university alumni. ChatGPT-3.5 (using a newly created account) answered all questions in Arabic. We compared the performance of ChatGPT including in the different fields of the exam; Obstetrics, Reproductive medicine and Infertility, Gynecology and Gynecologic Oncology, and also compared ChatGPT Arabic performance vs. previously published English medical tests.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>\n                      Overall, 123 authentic questions were analyzed. ChatGPT correctly answered 54 questions (43.9%, 95% CI: 35.1% \u2013 52.7%) and reached a score below 50%. There was no difference in ChatGPT performance in the four different subjects of the exam: Gynecologic Oncology (61.5%, 95% CI: 35.1% \u2013 87.9%), Gynecology (44.0%, 95% CI: 24.5% \u2013 63.5%), Obstetrics (42.3%, 95% CI: 28.9% \u2013 55.7%), Reproductive medicine and Infertility (39.4%, 95% CI: 22.7% \u2013 56.1%),\n                      <jats:italic>p<\/jats:italic>\n                      \u2009=\u2009.579. In a comparison to ChatGPT performance in 9,091 English language questions in the field of medicine, the performance of Arabic ChatGPT was lower (43.9% in Arabic vs. 60.7% in English,\n                      <jats:italic>p<\/jats:italic>\n                      \u2009&lt;\u2009.001).\n                    <\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusions<\/jats:title>\n                    <jats:p>ChatGPT-3.5 answered correctly approximately 44% of Arabic OBGYN medical licensure exam questions. At the time of writing of this manuscript, considering the results of our analysis, ChatGPT-3.5 cannot be considered a reliable primary tool for exam preparation in Arabic. Further research and efforts should be made to improve ChatGPT performance in other languages besides English especially Arabic.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.1186\/s12909-026-08753-3","type":"journal-article","created":{"date-parts":[[2026,2,11]],"date-time":"2026-02-11T08:54:20Z","timestamp":1770800060000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Performance of ChatGPT in Israeli Arabic-language OBGYN national medical licensure exam"],"prefix":"10.1186","volume":"26","author":[{"given":"Adiel","family":"Cohen","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Elior","family":"Eliasi","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Raanan","family":"Meyer","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yoav","family":"Brezinov","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Gabriel","family":"Levin","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2026,2,11]]},"reference":[{"issue":"1","key":"8753_CR1","doi-asserted-by":"publisher","first-page":"8","DOI":"10.1186\/2045-4015-2-8","volume":"2","author":"M Borow","year":"2013","unstructured":"Borow M, Levi B, Glekin M. Regulatory tasks of national medical associations - international comparison and the Israeli case. Isr J Health Policy Res. 2013;2(1):8.","journal-title":"Isr J Health Policy Res"},{"issue":"11","key":"8753_CR2","doi-asserted-by":"publisher","first-page":"984","DOI":"10.3109\/01421590903092426","volume":"31","author":"S Reis","year":"2009","unstructured":"Reis S, Borkan JM, Weingarten M. The current state of basic medical education in Israel: implications for a new medical school. Med Teach. 2009;31(11):984\u20139.","journal-title":"Med Teach"},{"issue":"1","key":"8753_CR3","doi-asserted-by":"publisher","first-page":"28","DOI":"10.1186\/s13584-023-00576-7","volume":"12","author":"Y Angel","year":"2023","unstructured":"Angel Y, Fire G. Healthcare policy changes in an era of health workforce shortage. Isr J Health Policy Res. 2023;12(1):28.","journal-title":"Isr J Health Policy Res"},{"key":"8753_CR4","doi-asserted-by":"publisher","first-page":"167","DOI":"10.2147\/AMEP.S287926","volume":"12","author":"TK Soong","year":"2021","unstructured":"Soong TK, Ho CM. Artificial intelligence in medical OSCEs: reflections and future developments. Adv Med Educ Pract. 2021;12:167\u201373.","journal-title":"Adv Med Educ Pract"},{"issue":"5","key":"8753_CR5","doi-asserted-by":"publisher","first-page":"e0268331","DOI":"10.1371\/journal.pone.0268331","volume":"17","author":"M Seer","year":"2022","unstructured":"Seer M, et al. Use of digital teaching resources and predictors of medical student performance during the pandemic: a prospective study. PLoS ONE. 2022;17(5):e0268331.","journal-title":"PLoS ONE"},{"issue":"3","key":"8753_CR6","doi-asserted-by":"publisher","first-page":"378","DOI":"10.1111\/1471-0528.17641","volume":"131","author":"G Levin","year":"2024","unstructured":"Levin G, et al. Performance of ChatGPT in medical examinations: A systematic review and a meta-analysis. BJOG. 2024;131(3):378\u201380.","journal-title":"BJOG"},{"issue":"7947","key":"8753_CR7","doi-asserted-by":"publisher","first-page":"224","DOI":"10.1038\/d41586-023-00288-7","volume":"614","author":"EAM van Dis","year":"2023","unstructured":"van Dis EAM, et al. ChatGPT: five priorities for research. Nature. 2023;614(7947):224\u20136.","journal-title":"Nature"},{"key":"8753_CR8","doi-asserted-by":"crossref","unstructured":"ChatGPT, Available at:\u00a0https:\/\/chatgpt.com. Accessed 1st Dec 2025.","DOI":"10.1177\/00336882251399226"},{"issue":"1","key":"8753_CR9","doi-asserted-by":"publisher","first-page":"2181052","DOI":"10.1080\/10872981.2023.2181052","volume":"28","author":"TB Arif","year":"2023","unstructured":"Arif TB, Munaf U, Ul-Haque I. The future of medical education and research: is ChatGPT a blessing or blight in disguise? Med Educ Online. 2023;28(1):2181052.","journal-title":"Med Educ Online"},{"issue":"1","key":"8753_CR10","doi-asserted-by":"publisher","first-page":"99","DOI":"10.1111\/1471-0528.17929","volume":"132","author":"G Levin","year":"2025","unstructured":"Levin G, et al. ChatGPT in a gynaecologic oncology multidisciplinary team tumour board: a feasibility study. BJOG. 2025;132(1):99\u2013101.","journal-title":"BJOG"},{"key":"8753_CR11","doi-asserted-by":"publisher","first-page":"e45312","DOI":"10.2196\/45312","volume":"9","author":"A Gilson","year":"2023","unstructured":"Gilson A, et al. How does ChatGPT perform on the United States Medical Licensing Examination? The implications of large language models for medical education and knowledge assessment. JMIR Med Educ. 2023;9:e45312.","journal-title":"JMIR Med Educ"},{"issue":"2","key":"8753_CR12","doi-asserted-by":"publisher","first-page":"e0000198","DOI":"10.1371\/journal.pdig.0000198","volume":"2","author":"TH Kung","year":"2023","unstructured":"Kung TH, et al. Performance of ChatGPT on USMLE: potential for AI-assisted medical education using large language models. PLoS Digit Health. 2023;2(2):e0000198.","journal-title":"PLoS Digit Health"},{"issue":"1","key":"8753_CR13","doi-asserted-by":"publisher","first-page":"198","DOI":"10.26719\/2009.15.1.198","volume":"15","author":"AA Mohamedani","year":"2009","unstructured":"Mohamedani AA, Abdul Rahman SH. A critical view of Arabic curriculum as a requirement for medical schools, Gezira University. East Mediterr Health J. 2009;15(1):198\u2013208.","journal-title":"East Mediterr Health J"},{"issue":"NP12","key":"8753_CR14","doi-asserted-by":"publisher","first-page":"1085","DOI":"10.1093\/asj\/sjad130","volume":"NP43","author":"P Humar","year":"2023","unstructured":"Humar P, et al. ChatGPT is equivalent to first year plastic surgery residents: evaluation of ChatGPT on the plastic surgery in-service exam. Aesthet Surg J. 2023;NP43(NP12):1085\u20139.","journal-title":"Aesthet Surg J."},{"issue":"12","key":"8753_CR15","doi-asserted-by":"publisher","first-page":"NP1078","DOI":"10.1093\/asj\/sjad128","volume":"43","author":"R Gupta","year":"2023","unstructured":"Gupta R, et al. Performance of ChatGPT on the plastic surgery inservice training examination. Aesthet Surg J. 2023;43(12):NP1078\u201382.","journal-title":"Aesthet Surg J"},{"issue":"6","key":"8753_CR16","doi-asserted-by":"publisher","first-page":"589","DOI":"10.1001\/jamaophthalmol.2023.1144","volume":"141","author":"A Mihalache","year":"2023","unstructured":"Mihalache A, Popovic MM, Muni RH. Performance of an artificial intelligence chatbot in ophthalmic knowledge assessment. JAMA Ophthalmol. 2023;141(6):589\u201397.","journal-title":"JAMA Ophthalmol"},{"key":"8753_CR17","doi-asserted-by":"publisher","first-page":"e47737","DOI":"10.2196\/47737","volume":"9","author":"P Giannos","year":"2023","unstructured":"Giannos P, Delardas O. Performance of ChatGPT on UK standardized admission tests: insights from the BMAT, TMUA, LNAT, and TSA examinations. JMIR Med Educ. 2023;9:e47737.","journal-title":"JMIR Med Educ"},{"issue":"8","key":"8753_CR18","doi-asserted-by":"publisher","first-page":"571","DOI":"10.1089\/dia.2023.0134","volume":"25","author":"A Nakhleh","year":"2023","unstructured":"Nakhleh A, Spitzer S, Shehadeh N. ChatGPT\u2019s response to the diabetes knowledge questionnaire: implications for diabetes education. Diabetes Technol Ther. 2023;25(8):571\u20133.","journal-title":"Diabetes Technol Ther"},{"issue":"2","key":"8753_CR19","doi-asserted-by":"publisher","first-page":"270","DOI":"10.1152\/advan.00036.2023","volume":"47","author":"M Subramani","year":"2023","unstructured":"Subramani M, Jaleel I, Krishna Mohan S. Evaluating the performance of ChatGPT in medical physiology university examination of phase I MBBS. Adv Physiol Educ. 2023;47(2):270\u20131.","journal-title":"Adv Physiol Educ"},{"issue":"3","key":"8753_CR20","doi-asserted-by":"publisher","first-page":"904","DOI":"10.3171\/2023.2.JNS23419","volume":"139","author":"BS Hopkins","year":"2023","unstructured":"Hopkins BS, et al. ChatGPT versus the neurosurgical written boards: a comparative analysis of artificial intelligence\/machine learning performance on neurosurgical board-style questions. J Neurosurg. 2023;139(3):904\u201311.","journal-title":"J Neurosurg"},{"key":"8753_CR21","doi-asserted-by":"publisher","first-page":"109732","DOI":"10.1016\/j.resuscitation.2023.109732","volume":"185","author":"N Fija\u010dko","year":"2023","unstructured":"Fija\u010dko N, et al. Can ChatGPT pass the life support exams without entering the American Heart Association course? Resuscitation. 2023;185:109732.","journal-title":"Resuscitation"},{"key":"8753_CR22","first-page":"1","volume":"20","author":"S Huh","year":"2023","unstructured":"Huh S. Are ChatGPT\u2019s knowledge and interpretation ability comparable to those of medical students in Korea for taking a parasitology examination?: a descriptive study. J Educ Eval Health Prof. 2023;20:1.","journal-title":"J Educ Eval Health Prof"},{"issue":"8","key":"8753_CR23","doi-asserted-by":"publisher","first-page":"1623","DOI":"10.1097\/CORR.0000000000002704","volume":"481","author":"ZC Lum","year":"2023","unstructured":"Lum ZC. Can artificial intelligence pass the American Board of Orthopaedic Surgery Examination? Orthopaedic residents versus ChatGPT. Clin Orthop Relat Res. 2023;481(8):1623\u201330.","journal-title":"Clin Orthop Relat Res"},{"issue":"12","key":"8753_CR24","doi-asserted-by":"publisher","first-page":"2280","DOI":"10.14309\/ajg.0000000000002320","volume":"118","author":"K Suchman","year":"2023","unstructured":"Suchman K, Garg S, Trindade AJ. ChatGPT fails the multiple-choice American College of gastroenterology self-assessment test. Am J Gastroenterol. 2023;118(12):2280\u20132.","journal-title":"Am J Gastroenterol"},{"issue":"2","key":"8753_CR25","doi-asserted-by":"publisher","first-page":"e34","DOI":"10.1016\/j.bja.2023.04.025","volume":"131","author":"L Birkett","year":"2023","unstructured":"Birkett L, Fowler T, Pullen S. Performance of ChatGPT on a primary FRCA multiple choice question bank. Br J Anaesth. 2023;131(2):e34\u20135.","journal-title":"Br J Anaesth"},{"issue":"2","key":"8753_CR26","doi-asserted-by":"publisher","first-page":"e31","DOI":"10.1016\/j.bja.2023.04.017","volume":"131","author":"D Shay","year":"2023","unstructured":"Shay D, et al. Assessment of ChatGPT success with specialty medical knowledge using anaesthesiology board examination practice questions. Br J Anaesth. 2023;131(2):e31\u20134.","journal-title":"Br J Anaesth"},{"issue":"5","key":"8753_CR27","doi-asserted-by":"publisher","first-page":"e230582","DOI":"10.1148\/radiol.230582","volume":"307","author":"R Bhayana","year":"2023","unstructured":"Bhayana R, Krishna S, Bleakney RR. Performance of ChatGPT on a radiology board-style examination: insights into current strengths and limitations. Radiology. 2023;307(5):e230582.","journal-title":"Radiology"},{"key":"8753_CR28","doi-asserted-by":"publisher","unstructured":"Deng,J., & Lin, Y, The benefits and challenges of ChatGPT: an overview. Frontiers in Computing and Intelligent Systems. 2023. https:\/\/doi.org\/10.54097\/fcis.v2i2.4465.","DOI":"10.54097\/fcis.v2i2.4465"},{"issue":"6","key":"8753_CR29","doi-asserted-by":"publisher","first-page":"100936","DOI":"10.1016\/j.ajogmf.2023.100936","volume":"5","author":"G Levin","year":"2023","unstructured":"Levin G, et al. Identifying ChatGPT-written OBGYN abstracts using a simple tool. Am J Obstet Gynecol MFM. 2023;5(6):100936.","journal-title":"Am J Obstet Gynecol MFM"},{"issue":"8","key":"8753_CR30","doi-asserted-by":"publisher","first-page":"100993","DOI":"10.1016\/j.ajogmf.2023.100993","volume":"5","author":"G Levin","year":"2023","unstructured":"Levin G, et al. ChatGPT-written OBGYN abstracts fool practitioners. Am J Obstet Gynecol MFM. 2023;5(8):100993.","journal-title":"Am J Obstet Gynecol MFM"},{"issue":"2","key":"8753_CR31","doi-asserted-by":"publisher","first-page":"e22331","DOI":"10.1148\/radiol.223312","volume":"307","author":"S Biswas","year":"2023","unstructured":"Biswas S. ChatGPT and the future of medical writing. Radiology. 2023;307(2):e22331.","journal-title":"Radiology"}],"container-title":["BMC Medical Education"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/article\/10.1186\/s12909-026-08753-3","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s12909-026-08753-3.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s12909-026-08753-3.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,6,16]],"date-time":"2026-06-16T18:04:59Z","timestamp":1781633099000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1186\/s12909-026-08753-3"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,2,11]]},"references-count":31,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2026,12]]}},"alternative-id":["8753"],"URL":"https:\/\/doi.org\/10.1186\/s12909-026-08753-3","relation":{},"ISSN":["1472-6920"],"issn-type":[{"value":"1472-6920","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,2,11]]},"assertion":[{"value":"1 October 2024","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"30 January 2026","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"11 February 2026","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"We used American Association for Public Opinion Research (AAPOR) reporting guidelines. This study did not require ethics approval, as we used only publicly accessible data, and no human participants were involved. No access to confidential or unpublished examination material occurred.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethics approval and consent to participate"}},{"value":"Not applicable.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent for publication"}},{"value":"The authors declare no competing interests.","order":4,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"432"}}