{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,1]],"date-time":"2026-10-01T19:41:01Z","timestamp":1790883661307,"version":"4.1.0"},"reference-count":34,"publisher":"SAGE Publications","license":[{"start":{"date-parts":[[2025,9,1]],"date-time":"2025-09-01T00:00:00Z","timestamp":1756684800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by-nc\/4.0\/"},{"start":{"date-parts":[[2025,9,1]],"date-time":"2025-09-01T00:00:00Z","timestamp":1756684800000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/journals.sagepub.com\/page\/policies\/text-and-data-mining-license"}],"content-domain":{"domain":["journals.sagepub.com"],"crossmark-restriction":true},"short-container-title":["Journal of Medical Education and Curricular Development"],"published-print":{"date-parts":[[2025,9]]},"abstract":"<jats:sec>\n            <jats:title>Introduction<\/jats:title>\n            <jats:p>ChatGPT has shown remarkable performance in medical licensing examinations such as the United States Medical Licensing Examination. However, limited research exists regarding its performance on national medical licensing exams in low-income countries. In Nepal, where nearly half of the candidates fail the national medical licensing exam, ChatGPT has the potential to contribute to medical education.<\/jats:p>\n          <\/jats:sec>\n          <jats:sec>\n            <jats:title>Objective<\/jats:title>\n            <jats:p>To evaluate ChatGPT's (GPT-4) performance on the Nepal Medical Council Licensing Medical Examination (NMCLE).<\/jats:p>\n          <\/jats:sec>\n          <jats:sec>\n            <jats:title>Methods<\/jats:title>\n            <jats:p>The NMCLE-May 2024 dataset, comprising 900 multiple-choice questions, was used to assess ChatGPT's performance. After excluding 8 questions that contained figures or were not compatible with text-only input, 892 questions were analyzed. Specific prompt, including a background description, question, and choices, was entered. The response generated by ChatGPT was compared taking responses from experienced clinicians as a reference. Descriptive statistics were used to present the results, and regression analysis was employed to determine the association between variables, including set, question type, pattern, and subject, and incorrect responses.<\/jats:p>\n          <\/jats:sec>\n          <jats:sec>\n            <jats:title>Results<\/jats:title>\n            <jats:p>GPT-4 generated 783 correct responses in 892 questions, an accuracy rate of 87.8%. Incorrect responses were more likely with questions requiring logical reasoning (odds ratio 14.7, 95% confidence interval [CI] 8.94-24.16).<\/jats:p>\n          <\/jats:sec>\n          <jats:sec>\n            <jats:title>Conclusions<\/jats:title>\n            <jats:p>ChatGPT-4 performs at a standard comparable to or above that of medical graduates on the Nepalese undergraduate medical licensing examination. Incorrect responses were mainly in questions requiring logical reasoning, underscoring the need for caution when relying on its outputs in the same. These findings are encouraging and highlight the need for further studies to evaluate its role as an educational resource in Nepalese medical education.<\/jats:p>\n          <\/jats:sec>","DOI":"10.1177\/23821205251384836","type":"journal-article","created":{"date-parts":[[2025,10,6]],"date-time":"2025-10-06T17:22:39Z","timestamp":1759771359000},"update-policy":"https:\/\/doi.org\/10.1177\/sage-journals-update-policy","source":"Crossref","is-referenced-by-count":3,"title":["Performance of ChatGPT-4 on the Nepalese Undergraduate Medical Licensing Examination: A Cross-Sectional Study"],"prefix":"10.1177","volume":"12","author":[{"given":"Prajjwol","family":"Luitel","sequence":"first","affiliation":[{"name":"Maharajgunj Medical Campus, Institute of Medicine, Tribhuvan University, Kathmandu, Nepal"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sujan","family":"Paudel","sequence":"additional","affiliation":[{"name":"Maharajgunj Medical Campus, Institute of Medicine, Tribhuvan University, Kathmandu, Nepal"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Devansh","family":"Upadhya","sequence":"additional","affiliation":[{"name":"Maharajgunj Medical Campus, Institute of Medicine, Tribhuvan University, Kathmandu, Nepal"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0006-7985-0076","authenticated-orcid":false,"given":"Amit","family":"Yadav","sequence":"additional","affiliation":[{"name":"Maharajgunj Medical Campus, Institute of Medicine, Tribhuvan University, Kathmandu, Nepal"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Gehendra Jung","family":"Kunwar","sequence":"additional","affiliation":[{"name":"Maharajgunj Medical Campus, Institute of Medicine, Tribhuvan University, Kathmandu, Nepal"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"179","published-online":{"date-parts":[[2025,10,6]]},"reference":[{"key":"e_1_3_3_2_2","unstructured":"Performance of ChatGPT on Chinese national medical licensing examinations: a five-year examination evaluation study for physicians pharmacists and nurses | BMC Medical Education | Full Text [Internet]. Cited May 11 2025. https:\/\/bmcmededuc.biomedcentral.com\/articles\/10.1186\/s12909-024-05125-7"},{"key":"e_1_3_3_3_2","doi-asserted-by":"publisher","DOI":"10.2196\/57896"},{"key":"e_1_3_3_4_2","article-title":"ChatGPT: effective writing is succinct","volume":"381","author":"Kovoor JG","year":"2023","unstructured":"Kovoor JG, Gupta AK, Bacchi S. ChatGPT: effective writing is succinct. Br Med J. 2023;381:p1125.","journal-title":"Br Med J."},{"key":"e_1_3_3_5_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.oor.2024.100511"},{"key":"e_1_3_3_6_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.oor.2024.100289"},{"key":"e_1_3_3_7_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.oor.2024.100227"},{"key":"e_1_3_3_8_2","unstructured":"AI-dependency in scientific writing \u2013 ScienceDirect [Internet]. Cited May 11 2025. https:\/\/www.sciencedirect.com\/science\/article\/pii\/S2772906024001158?via%3Dihub"},{"key":"e_1_3_3_9_2","doi-asserted-by":"publisher","DOI":"10.1080\/0142159X.2023.2249588"},{"key":"e_1_3_3_10_2","doi-asserted-by":"publisher","DOI":"10.1148\/radiol.230582"},{"key":"e_1_3_3_11_2","doi-asserted-by":"publisher","DOI":"10.1093\/ckj\/sfae193"},{"key":"e_1_3_3_12_2","doi-asserted-by":"publisher","DOI":"10.1093\/asj\/sjad130"},{"key":"e_1_3_3_13_2","doi-asserted-by":"publisher","DOI":"10.1097\/JCMA.0000000000000946"},{"key":"e_1_3_3_14_2","doi-asserted-by":"publisher","DOI":"10.2196\/48039"},{"key":"e_1_3_3_15_2","unstructured":"Nepal Medical Council reveals a 56.68% pass rate in MBBS and BDS registration exams [Internet]. Edusanjal. Cited May 11 2025. https:\/\/edusanjal.com\/news\/nepal-medical-council-reveals-a-5668-pass-rate-in-mbbs-and-bds-registration-exams\/"},{"key":"e_1_3_3_16_2","unstructured":"Patel H Parmar S. Prompt engineering for large language model. 2024."},{"key":"e_1_3_3_17_2","doi-asserted-by":"publisher","DOI":"10.2196\/45312"},{"key":"e_1_3_3_18_2","doi-asserted-by":"publisher","DOI":"10.1136\/bmj.39335.541782.AD"},{"key":"e_1_3_3_19_2","doi-asserted-by":"publisher","DOI":"10.2196\/49964"},{"key":"e_1_3_3_20_2","doi-asserted-by":"crossref","unstructured":"Yang Z Yao Z Tasmin M et al. Performance of multimodal GPT-4V on USMLE with image: potential for imaging diagnostic support with explanations. medRxiv (Cold Spring Harbor Laboratory); 2023. https:\/\/doi.org\/10.1101\/2023.10.26.23297629","DOI":"10.1101\/2023.10.26.23297629"},{"key":"e_1_3_3_21_2","doi-asserted-by":"publisher","DOI":"10.14309\/ajg.0000000000002320"},{"issue":"2","key":"e_1_3_3_22_2","article-title":"\u00bfEs capaz GPT-4 de aprobar el MIR 2023? Comparativa entre GPT-4 y ChatGPT-3 en los ex\u00e1menes MIR 2022 y 2023","volume":"5","author":"Cerame A","year":"2024","unstructured":"Cerame A, Juaneda J, Estrella-Porter P, et al. \u00bfEs capaz GPT-4 de aprobar el MIR 2023? Comparativa entre GPT-4 y ChatGPT-3 en los ex\u00e1menes MIR 2022 y 2023. Rev Esp Educ M\u00e9dica. 2024;5(2). https:\/\/doi.org\/10.6018\/edumed.604091","journal-title":"Rev Esp Educ M\u00e9dica"},{"key":"e_1_3_3_23_2","doi-asserted-by":"crossref","unstructured":"Kaneda Y Tanimoto T Ozaki A Sato T Takahashi K. Can ChatGPT Pass the 2023 Japanese National Medical Licensing Examination [Internet]. 2023. Cited May 11 2025.","DOI":"10.20944\/preprints202303.0191.v1"},{"key":"e_1_3_3_24_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10916-023-01961-0"},{"issue":"1","key":"e_1_3_3_25_2","article-title":"Performance of ChatGPT across different versions in medical licensing examinations worldwide: systematic review and meta-analysis","volume":"26","author":"Liu M","year":"2024","unstructured":"Liu M, Okuhara T, Chang X, et al. Performance of ChatGPT across different versions in medical licensing examinations worldwide: systematic review and meta-analysis. J Med Internet Res. 2024;26(1):e60807.","journal-title":"J Med Internet Res"},{"key":"e_1_3_3_26_2","first-page":"2333794X","article-title":"ChatGPT yields a passing score on a pediatric board preparatory exam but raises red flags","volume":"11","author":"Le M","year":"2024","unstructured":"Le M, Davis M. ChatGPT yields a passing score on a pediatric board preparatory exam but raises red flags. Glob Pediatr Health. 2024;11:2333794X\u2013241240327.","journal-title":"Glob Pediatr Health"},{"key":"e_1_3_3_27_2","unstructured":"Alessi MR Gomes HA De Castro ML Okamoto CT. Performance of ChatGPT in solving questions from the Progress Test (Brazilian National Medical Exam): a potential artificial intelligence tool in medical practice. Cureus 2024;16(7):e64924."},{"key":"e_1_3_3_28_2","doi-asserted-by":"publisher","DOI":"10.1093\/jbmr\/zjad007"},{"issue":"2","key":"e_1_3_3_29_2","first-page":"276","article-title":"Assessing the capability of ChatGPT, Google Bard, and Microsoft Bing in solving radiology case vignettes","volume":"34","author":"Sarangi PK","year":"2023","unstructured":"Sarangi PK, Narayan RK, Mohakud S, Vats A, Sahani D, Mondal H. Assessing the capability of ChatGPT, Google Bard, and Microsoft Bing in solving radiology case vignettes. Indian J Radiol Imaging. 2023;34(2):276.","journal-title":"Indian J Radiol Imaging"},{"key":"e_1_3_3_30_2","doi-asserted-by":"publisher","DOI":"10.1055\/s-0044-1782165"},{"issue":"3","key":"e_1_3_3_31_2","article-title":"Evaluation of ChatGPT\u2019s performance in providing treatment recommendations for pediatric diseases","volume":"1","author":"Wei Q","year":"2023","unstructured":"Wei Q, Wang Y, Yao Z, et al. Evaluation of ChatGPT\u2019s performance in providing treatment recommendations for pediatric diseases. Pediatr Discov. 2023;1(3):e42.","journal-title":"Pediatr Discov"},{"key":"e_1_3_3_32_2","unstructured":"Assessment of the capacity of ChatGPT as a self-learning tool in medical pharmacology: a study using MCQs | BMC Medical Education | Full Text [Internet]. Cited May 11 2025. https:\/\/bmcmededuc.biomedcentral.com\/articles\/10.1186\/s12909-023-04832-x"},{"issue":"1","key":"e_1_3_3_33_2","first-page":"35","article-title":"Exploring radiology postgraduate students\u2019 engagement with large language models for educational purposes: a study of knowledge, attitudes, and practices","volume":"35","author":"Sarangi PK","year":"2024","unstructured":"Sarangi PK, Panda BB, Sanjay P, Pattanayak D, Panda S, Mondal H. Exploring radiology postgraduate students\u2019 engagement with large language models for educational purposes: a study of knowledge, attitudes, and practices. Indian J Radiol Imaging. 2024;35(1):35.","journal-title":"Indian J Radiol Imaging"},{"issue":"2","key":"e_1_3_3_34_2","first-page":"72","article-title":"Barriers of critical thinking in medical students\u2019 curriculum from the viewpoint of medical education experts: a qualitative study","volume":"8","author":"Kasalaei A","year":"2020","unstructured":"Kasalaei A, Amini M, Nabeiei P, Bazrafkan L, Mousavinezhad H. Barriers of critical thinking in medical students\u2019 curriculum from the viewpoint of medical education experts: a qualitative study. J Adv Med Educ Prof. 2020;8(2):72\u201382.","journal-title":"J Adv Med Educ Prof"},{"key":"e_1_3_3_35_2","doi-asserted-by":"publisher","DOI":"10.1055\/s-0044-1787974"}],"container-title":["Journal of Medical Education and Curricular Development"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/23821205251384836","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/full-xml\/10.1177\/23821205251384836","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/23821205251384836","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,8]],"date-time":"2025-10-08T13:17:31Z","timestamp":1759929451000},"score":1,"resource":{"primary":{"URL":"https:\/\/journals.sagepub.com\/doi\/10.1177\/23821205251384836"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,9]]},"references-count":34,"alternative-id":["10.1177\/23821205251384836"],"URL":"https:\/\/doi.org\/10.1177\/23821205251384836","relation":{},"ISSN":["2382-1205","2382-1205"],"issn-type":[{"value":"2382-1205","type":"print"},{"value":"2382-1205","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,9]]},"article-number":"23821205251384836"}}