{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,10]],"date-time":"2026-10-10T21:41:35Z","timestamp":1791668495423,"version":"4.3.4"},"reference-count":10,"publisher":"Ovid Technologies (Wolters Kluwer Health)","issue":"2","license":[{"start":{"date-parts":[[2026,4,1]],"date-time":"2026-04-01T00:00:00Z","timestamp":1775001600000},"content-version":"unspecified","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by-nc-nd\/4.0"}],"content-domain":{"domain":["lww.com","ovid.com"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2026,4]]},"abstract":"<jats:title>Abstract:<\/jats:title>\n                  <jats:sec>\n                    <jats:title>OBJECTIVES:<\/jats:title>\n                    <jats:p>We aimed to evaluate the performance of a large language model (ChatGPT) in answering official sample questions from the Turkish Board of Emergency Medicine (TBEM). Two versions of the model, GPT-4 and GPT-4o, were assessed to explore consistency and accuracy across iterations.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>METHODS:<\/jats:title>\n                    <jats:p>A cross-sectional observational study was conducted using 25 standardized multiple-choice questions publicly released by TBEM. Each question was manually entered into GPT-4 and GPT-4o through the OpenAI interface. Both models were prompted to select the best single answer from the provided options without additional clarification or training context. Model responses were evaluated for accuracy, consistency upon repetition, and domain-specific error types. This study is compliant with the STROBE statement and the MedinAI reporting guidelines.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>RESULTS:<\/jats:title>\n                    <jats:p>GPT-4 correctly answered 20 out of 25 questions (80%) on the first attempt. On repetition, its score improved to 84%. GPT-4o also achieved a score of 88% (22\/25) on its first attempt and showed consistent results upon a second evaluation, providing identical answers in both trials. Errors occurred in the domains of trauma during pregnancy, pediatric resuscitation, and adult resuscitation protocols. Both models demonstrated strong performance in fact-based domains and in questions involving image descriptions.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>CONCLUSION:<\/jats:title>\n                    <jats:p>GPT-4 and GPT-4o performed above the TBEM passing threshold, showing solid accuracy across a range of emergency medicine topics. Both excelled in fact-based and image-related questions. However, they showed limitations in clinical reasoning, particularly in scenarios requiring nuanced judgment. These tools may support examination preparation but should not replace the expertise of trained emergency physicians.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.4103\/tjem.tjem_262_25","type":"journal-article","created":{"date-parts":[[2026,4,3]],"date-time":"2026-04-03T09:00:09Z","timestamp":1775206809000},"page":"110-115","update-policy":"https:\/\/doi.org\/10.1097\/lww.0000000000001000","source":"Crossref","is-referenced-by-count":1,"title":["Evaluation of ChatGPT\u2019s performance on emergency medicine board examination questions"],"prefix":"10.4103","volume":"26","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-4141-2661","authenticated-orcid":false,"given":"Mustafa Can","family":"Guzelce","sequence":"first","affiliation":[{"name":"Department of Emergency Medicine, Izmir University of Economics Medicalpoint Hospital, Izmir, T\u00fcrkiye"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7908-6422","authenticated-orcid":false,"given":"Sefer","family":"Ozgur","sequence":"additional","affiliation":[{"name":"Department of Emergency Medicine, Izmir University of Economics Medicalpoint Hospital, Izmir, T\u00fcrkiye"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8954-8421","authenticated-orcid":false,"given":"Ilker","family":"Salli","sequence":"additional","affiliation":[{"name":"Department of Emergency Medicine, Izmir Tepecik Training and Research Hospital, Izmir, T\u00fcrkiye"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"276","published-online":{"date-parts":[[2026,4,3]]},"reference":[{"key":"R2-20260414","doi-asserted-by":"crossref","first-page":"e45312","DOI":"10.2196\/45312","article-title":"How does ChatGPT perform on the United States Medical Licensing Examination (USMLE)?The implications of large language models for medical education and knowledge assessment","volume":"9","author":"Gilson","year":"2023","journal-title":"JMIR Med Educ"},{"key":"R3-20260414","doi-asserted-by":"crossref","first-page":"945","DOI":"10.1007\/s11096-025-01905-3","article-title":"Guidelines for reporting artificial intelligence studies in medicines, pharmacotherapy, and pharmaceutical services:MedinAI development, validation and statement","volume":"47","author":"Bottacin","year":"2025","journal-title":"Int J Clin Pharm"},{"key":"R4-20260414","doi-asserted-by":"crossref","first-page":"e67696","DOI":"10.2196\/67696","article-title":"Evaluation of ChatGPT performance on emergency medicine board examination questions:Observational Study","volume":"4","author":"Pastrak","year":"2025","journal-title":"JMIR AI"},{"key":"R5-20260414","doi-asserted-by":"crossref","first-page":"e37325","DOI":"10.1097\/MD.0000000000037325","article-title":"Comparison of the problem-solving performance of ChatGPT-3.5, ChatGPT-4, Bing chat, and bard for the Korean emergency medicine board examination question bank","volume":"103","author":"Lee","year":"2024","journal-title":"Medicine (Baltimore)"},{"key":"R6-20260414","doi-asserted-by":"crossref","first-page":"1959","DOI":"10.1007\/s00330-024-11035-5","article-title":"Assessing GPT-4 multimodal performance in radiological image analysis","volume":"35","author":"Brin","year":"2025","journal-title":"Eur Radiol"},{"key":"R7-20260414","doi-asserted-by":"crossref","first-page":"61","DOI":"10.5339\/qmj.2024.61","article-title":"Performance of ChatGPT in emergency medicine residency exams in Qatar:A comparative analysis with resident physicians","volume":"2024","author":"Iftikhar","year":"2024","journal-title":"Qatar Med J"},{"key":"R8-20260414","doi-asserted-by":"crossref","first-page":"e64284","DOI":"10.2196\/64284","article-title":"Performance evaluation and implications of large language models in radiology board exams:Prospective comparative analysis","volume":"11","author":"Wei","year":"2025","journal-title":"JMIR Med Educ"},{"key":"R9-20260414","doi-asserted-by":"crossref","first-page":"e000530","DOI":"10.1136\/bmjno-2023-000530","article-title":"Assessment of ChatGPT's performance on neurology written board examination questions","volume":"5","author":"Chen","year":"2023","journal-title":"BMJ Neurol Open"},{"key":"R10-20260414","doi-asserted-by":"crossref","first-page":"e56128","DOI":"10.2196\/56128","article-title":"Assessment of ChatGPT-4 in family medicine board examinations using advanced AI learning and analytical methods:Observational study","volume":"10","author":"Goodings","year":"2024","journal-title":"JMIR Med Educ"},{"key":"R11-20260414","doi-asserted-by":"crossref","first-page":"3547","DOI":"10.57187\/s.3547","article-title":"Experimental assessment of the performance of artificial intelligence in solving multiple-choice board exams in cardiology","volume":"154","author":"Huwiler","year":"2024","journal-title":"Swiss Med Wkly"}],"container-title":["Turkish Journal of Emergency Medicine"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.ovid.com\/10.4103\/tjem.tjem_262_25","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,8,18]],"date-time":"2026-08-18T15:43:57Z","timestamp":1787067837000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.ovid.com\/10.4103\/tjem.tjem_262_25"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,4]]},"references-count":10,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2026]]}},"URL":"https:\/\/doi.org\/10.4103\/tjem.tjem_262_25","relation":{},"ISSN":["2452-2473","2452-2473"],"issn-type":[{"value":"2452-2473","type":"print"},{"value":"2452-2473","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,4]]},"assertion":[{"value":"2025-07-19","name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-11-04","name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}}]}}