{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,28]],"date-time":"2026-08-28T11:28:54Z","timestamp":1787916534966,"version":"build-2784847793"},"reference-count":19,"publisher":"SAGE Publications","issue":"11","license":[{"start":{"date-parts":[[2025,5,12]],"date-time":"2025-05-12T00:00:00Z","timestamp":1747008000000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/journals.sagepub.com\/page\/policies\/text-and-data-mining-license"}],"content-domain":{"domain":["journals.sagepub.com"],"crossmark-restriction":true},"short-container-title":["The American Surgeon\u2122"],"published-print":{"date-parts":[[2025,11]]},"abstract":"<jats:sec>\n                    <jats:title>Objective<\/jats:title>\n                    <jats:p>This study aimed to evaluate the performance of large language models (LLMs) in answering questions from the American Board of Surgery In-Training Examination (ABSITE).<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Methods<\/jats:title>\n                    <jats:p>Multiple choice ABSITE Quiz was entered into the most popular LLMs as prompts. ChatGPT-4 (OpenAI), Copilot (Microsoft), and Gemini (Google) were used in the study. The research comprised 170 questions from 2017 to 2022, which were divided into four subgroups: Definitions, Biochemistry\/Pharmaceutical, Case Scenario, and Treatment &amp; Surgical Procedures. All questions were queried in LLMs, between October 1, 2024, and October 5, 2024. Correct answer rates of LLMs were evaluated.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>\n                      The correct response rates for all questions were 79.4% for ChatGPT, 77.6% for Copilot, and 52.9% for Gemini, with Gemini significantly lower than both LLMs (\n                      <jats:italic>P<\/jats:italic>\n                      &lt; 0.001). In the definition category, the correct response rates were 93.5% for ChatGPT, 90.3% for Copilot, and 64.5% for Gemini, with Gemini significantly lower (\n                      <jats:italic>P<\/jats:italic>\n                      = 0.005 and\n                      <jats:italic>P<\/jats:italic>\n                      = 0.015, respectively). In the Biochemistry\/Pharmaceutical question category, the correct response rates were equal in all three groups (83.3%). In the Case Scenario category, the correct response rates were 76.3% in ChatGPT, 72.8% for Copilot, and 46.5% for Gemini, with Gemini significantly lower (\n                      <jats:italic>P<\/jats:italic>\n                      &lt; 0.001). In the Treatment &amp; Surgical Procedures category, the correct response rates were 69.2% for ChatGPT, 84.6% for Copilot, and 53.8% for Gemini. Although Gemini had the lowest accuracy, there was no statistically significant difference (\n                      <jats:italic>P<\/jats:italic>\n                      = 0.236).\n                    <\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusion<\/jats:title>\n                    <jats:p>In the ABSITE Quiz, ChatGPT and Copilot had similar success, whereas Gemini was significantly behind.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.1177\/00031348251341956","type":"journal-article","created":{"date-parts":[[2025,5,12]],"date-time":"2025-05-12T07:38:47Z","timestamp":1747035527000},"page":"1923-1929","update-policy":"https:\/\/doi.org\/10.1177\/sage-journals-update-policy","source":"Crossref","is-referenced-by-count":7,"title":["Can American Board of Surgery in Training Examinations be passed by Large Language Models? Comparative assessment of Gemini, Copilot, and ChatGPT"],"prefix":"10.1177","volume":"91","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-1483-8176","authenticated-orcid":false,"given":"Ahmet Necati","family":"Sanli","sequence":"first","affiliation":[{"name":"Department of General Surgery, Abdulkadir Yuksel State Hospital, Gaziantep, Turkey"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6545-5757","authenticated-orcid":false,"given":"Deniz Esin","family":"Tekcan Sanli","sequence":"additional","affiliation":[{"name":"Department of Radiology, School of Medicine, Gazianep University, Gaziantep, Turkey"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6690-2152","authenticated-orcid":false,"given":"Ali","family":"Karabulut","sequence":"additional","affiliation":[{"name":"Department of General Surgery, Bagcilar Training and Research, University of Health Sciences, Istanbul, Turkey"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"179","published-online":{"date-parts":[[2025,5,12]]},"reference":[{"key":"e_1_3_4_2_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41591-018-0307-0"},{"key":"e_1_3_4_3_2","doi-asserted-by":"publisher","DOI":"10.7759\/cureus.65083"},{"key":"e_1_3_4_4_2","doi-asserted-by":"publisher","DOI":"10.3306\/AJHS.2024.39.01.23"},{"key":"e_1_3_4_5_2","doi-asserted-by":"publisher","DOI":"10.2196\/49970"},{"key":"e_1_3_4_6_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.xops.2023.100324"},{"key":"e_1_3_4_7_2","doi-asserted-by":"publisher","DOI":"10.1227\/neu.0000000000002632"},{"key":"e_1_3_4_8_2","doi-asserted-by":"publisher","DOI":"10.1227\/neu.0000000000002551"},{"key":"e_1_3_4_9_2","doi-asserted-by":"publisher","DOI":"10.4174\/astr.2023.104.5.269"},{"key":"e_1_3_4_10_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.bjps.2023.10.141"},{"key":"e_1_3_4_11_2","doi-asserted-by":"publisher","DOI":"10.1007\/s00345-024-05137-4"},{"key":"e_1_3_4_12_2","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pdig.0000198"},{"key":"e_1_3_4_13_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.soard.2024.04.014"},{"key":"e_1_3_4_14_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2024.04.060"},{"key":"e_1_3_4_15_2","unstructured":"GPT-4 technical report. https:\/\/cdn.openai.com\/papers\/gpt-4.pdf. Accessed [January 2 2025]."},{"key":"e_1_3_4_16_2","unstructured":"Copilot report. https:\/\/learn.microsoft.com\/en-us\/dynamics365\/customer-service\/use\/copilot-analytics-report#copilot-report. Accessed [January 2 2025]."},{"key":"e_1_3_4_17_2","unstructured":"Introducing Gemini: our largest and most capable AI model. https:\/\/blog.google\/technology\/ai\/google-gemini-ai\/#sundar-note. Accessed [January 2 2025]."},{"key":"e_1_3_4_18_2","doi-asserted-by":"publisher","unstructured":"Liu Y Cao J Liu C Ding K Jin L. Datasets for large language models: a comprehensive survey. arXiv preprint. 2024;2402.18041. doi: 10.48550\/arXiv.2402.18041","DOI":"10.48550\/arXiv.2402.18041"},{"key":"e_1_3_4_19_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.medp.2024.100030"},{"key":"e_1_3_4_20_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41746-024-01283-6"}],"container-title":["The American Surgeon\u2122"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/00031348251341956","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/full-xml\/10.1177\/00031348251341956","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/00031348251341956","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,4,28]],"date-time":"2026-04-28T12:37:48Z","timestamp":1777379868000},"score":1,"resource":{"primary":{"URL":"https:\/\/journals.sagepub.com\/doi\/10.1177\/00031348251341956"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,5,12]]},"references-count":19,"journal-issue":{"issue":"11","published-print":{"date-parts":[[2025,11]]}},"alternative-id":["10.1177\/00031348251341956"],"URL":"https:\/\/doi.org\/10.1177\/00031348251341956","relation":{},"ISSN":["0003-1348","1555-9823"],"issn-type":[{"value":"0003-1348","type":"print"},{"value":"1555-9823","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,5,12]]}}}