{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,10]],"date-time":"2026-08-10T04:29:30Z","timestamp":1786336170572,"version":"build-2736575974"},"reference-count":22,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2026,8,10]],"date-time":"2026-08-10T00:00:00Z","timestamp":1786320000000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2026,8,10]],"date-time":"2026-08-10T00:00:00Z","timestamp":1786320000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Arch Orthop Trauma Surg"],"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:sec>\n                    <jats:title>Introduction<\/jats:title>\n                    <jats:p>Artificial intelligence (AI) has evolved rapidly in recent years and is becoming increasingly integrated into many areas of medicine. In the medical field, these systems have attracted considerable attention because of their potential applications in clinical decision support, medical education, scientific communication, and postgraduate training. The aim of the present study was to evaluate whether ChatGPT could achieve a passing score on the Polish National Medical Specialization Examination (Panstwowy Egzamin Specjalizacyjny, PES) in orthopedics and traumatology and to determine how different prompting strategies influenced its performance.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Materials and methods<\/jats:title>\n                    <jats:p>Authors systematically assessed the performance of ChatGPT-4 across five consecutive official PES exams (from Autumn 2023 to Spring 2025) in orthopedics and traumatology. Each exam was administered using three distinct prompting strategies: Professor prompt, Specialist prompt, Resident prompt. For each exam session (e.g., Spring 2024, Autumn 2023), the prompts were submitted to ChatGPT sequentially. The model\u2019s responses were evaluated against the official answer key published by the Polish Center of Medical Exams (Centrum Egzamin\u00f3w Medycznych, CEM).<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>The results were averaged across all prompts. The overall accuracy was 81% (range: 68.33%\u201385.83%). The highest score (85.83%) was recorded multiple times across different prompt types, indicating that the model could approach or exceed the minimum passing threshold depending on prompt structure. Agreement between prompting strategies was moderate to substantial (Cohen\u2019s \u03ba 0.518\u20130.632; Fleiss\u2019 \u03ba\u2009=\u20090.571). Radiology-related questions demonstrated the highest error rate among all question categories.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusions<\/jats:title>\n                    <jats:p>Large language models such as ChatGPT can perform well on knowledge-based orthopedic examinations and may serve as useful tools for examination preparation and educational support. However, examination success should not be interpreted as evidence of clinical competence or readiness for independent surgical practice. Specialist certification and patient care remain dependent on human expertise, practical skills, and professional responsibility.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.1007\/s00402-026-06435-9","type":"journal-article","created":{"date-parts":[[2026,8,10]],"date-time":"2026-08-10T04:00:34Z","timestamp":1786334434000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Can ChatGPT pass the polish national medical specialization examination in orthopedics and traumatology?"],"prefix":"10.1007","volume":"146","author":[{"given":"Bartosz","family":"Maci\u0105g","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Krzysztof","family":"Bujak","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Dawid","family":"Jegierski","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Artur","family":"Stolarczyk","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2026,8,10]]},"reference":[{"issue":"2","key":"6435_CR1","doi-asserted-by":"publisher","first-page":"e0000198","DOI":"10.1371\/journal.pdig.0000198","volume":"2","author":"TH Kung","year":"2023","unstructured":"Kung TH, Cheatham M, Medenilla A, Sillos C, De Leon L, Elepano C et al (2023) Performance of ChatGPT on USMLE: Potential for AI-assisted medical education using large language models. PLoS Digit Health 2(2):e0000198. https:\/\/doi.org\/10.1371\/journal.pdig.0000198","journal-title":"PLoS Digit Health"},{"issue":"10","key":"6435_CR2","doi-asserted-by":"publisher","first-page":"1128","DOI":"10.1111\/medu.15402","volume":"58","author":"HC Lucas","year":"2024","unstructured":"Lucas HC, Tannoury J, Chbib N, Dagher M, El Hajj A, Mailhac A et al (2024) A systematic review of large language models and their implications in medical education. Med Educ 58(10):1128\u20131140. https:\/\/doi.org\/10.1111\/medu.15402","journal-title":"Med Educ"},{"key":"6435_CR3","doi-asserted-by":"publisher","first-page":"e60807","DOI":"10.2196\/60807","volume":"26","author":"M Liu","year":"2024","unstructured":"Liu M, Wang H, Wang G et al (2024) Performance of ChatGPT across different versions in medical licensing examinations worldwide: systematic review and meta-analysis. J Med Internet Res 26:e60807","journal-title":"J Med Internet Res"},{"key":"6435_CR4","doi-asserted-by":"publisher","first-page":"16495","DOI":"10.2196\/60807","volume":"13","author":"D Brin","year":"2023","unstructured":"Brin D, Sorin V, Vaid A et al (2023) Comparing ChatGPT and GPT-4 performance in USMLE soft skill assessments. Sci Rep 13:16495. https:\/\/doi.org\/10.2196\/60807","journal-title":"Sci Rep"},{"key":"6435_CR5","doi-asserted-by":"publisher","unstructured":"Kung JE, Cohn MR, Zhang LQ et al (2023) Evaluating ChatGPT performance on the Orthopaedic In-Training Examination. JBJS Open Access 8(3). https:\/\/doi.org\/10.2106\/JBJS.OA.23.00056. e23.00056","DOI":"10.2106\/JBJS.OA.23.00056"},{"issue":"3","key":"6435_CR6","doi-asserted-by":"publisher","first-page":"382","DOI":"10.4055\/cios23179","volume":"16","author":"SE Kim","year":"2024","unstructured":"Kim SE, Yoon BH, Park JH et al (2024) Performance of ChatGPT on solving orthopedic board-style questions: a comparative analysis of ChatGPT 3.5 and ChatGPT 4. Clin Orthop Surg 16(3):382\u2013389. https:\/\/doi.org\/10.4055\/cios23179","journal-title":"Clin Orthop Surg"},{"key":"6435_CR7","doi-asserted-by":"publisher","first-page":"e63399","DOI":"10.2196\/59607","volume":"10","author":"C Zhang","year":"2024","unstructured":"Zhang C, Zhang Z, Li Y et al (2024) Examining the role of large language models in orthopedics: systematic review. JMIR Med Educ 10:e63399. https:\/\/doi.org\/10.2196\/59607","journal-title":"JMIR Med Educ"},{"key":"6435_CR8","unstructured":"Ustawa z dnia 5 grudnia 1996 r. o zawodach lekarza i lekarza dentysty. Dz.U. 2023 poz. 1516"},{"key":"6435_CR9","unstructured":"Rozporzadzenie Ministra Zdrowia z dnia 4 maja 2023 r. w sprawie specjalizacji lekarzy i lekarzy dentystow. Dz.U. 2023 poz. 1771"},{"key":"6435_CR10","doi-asserted-by":"publisher","unstructured":"Nori H, King N, McKinney SM, Carignan D, Horvitz E Capabilities of GPT-4 on medical challenge problems. arXiv. 2023:2303.13375. https:\/\/doi.org\/10.48550\/arXiv.2303.13375","DOI":"10.48550\/arXiv.2303.13375"},{"key":"6435_CR11","doi-asserted-by":"publisher","first-page":"e45312","DOI":"10.2196\/45312","volume":"9","author":"A Gilson","year":"2023","unstructured":"Gilson A, Safranek CW, Huang T, Socrates V, Chi L, Taylor RA et al (2023) How does ChatGPT perform on the United States Medical Licensing Examination? The implications of large language models for medical education and knowledge assessment. JMIR Med Educ 9:e45312. https:\/\/doi.org\/10.2196\/45312","journal-title":"JMIR Med Educ"},{"key":"6435_CR12","doi-asserted-by":"publisher","first-page":"e60807","DOI":"10.2196\/60807","volume":"26","author":"M Liu","year":"2024","unstructured":"Liu M, Wang H, Wang G et al (2024) Performance of ChatGPT across different versions in medical licensing examinations worldwide: systematic review and meta-analysis. J Med Internet Res 26:e60807. https:\/\/doi.org\/10.2196\/60807","journal-title":"J Med Internet Res"},{"key":"6435_CR13","doi-asserted-by":"publisher","unstructured":"Kung JE, Cohn MR, Zhang LQ et al (2023) Evaluating ChatGPT performance on the Orthopaedic In-Training Examination. JBJS Open Access 8(3). https:\/\/doi.org\/10.2106\/JBJS.OA.23.00056. e23.00056","DOI":"10.2106\/JBJS.OA.23.00056"},{"key":"6435_CR14","doi-asserted-by":"publisher","DOI":"10.3928\/01477447-20230922-05","author":"HL Hofmann","year":"2024","unstructured":"Hofmann HL, Losurdo JP, Leland HA (2024) GPT-4\u2019s performance on orthopedic surgery board examination questions. J Surg Educ. https:\/\/doi.org\/10.3928\/01477447-20230922-05","journal-title":"J Surg Educ"},{"key":"6435_CR15","doi-asserted-by":"publisher","DOI":"10.4055\/cios23179","author":"SE Kim","year":"2024","unstructured":"Kim SE, Yoon BH, Park JH et al (2024) Performance of ChatGPT on solving orthopedic board-style questions: a comparative analysis of ChatGPT 3.5 and ChatGPT 4. Clin Orthop Surg. https:\/\/doi.org\/10.4055\/cios23179","journal-title":"Clin Orthop Surg"},{"key":"6435_CR16","doi-asserted-by":"publisher","first-page":"e59607","DOI":"10.2196\/59607","volume":"10","author":"C Zhang","year":"2024","unstructured":"Zhang C, Zhang Z, Li Y et al (2024) Examining the role of large language models in orthopedics: opportunities, applications, and challenges. JMIR Med Educ 10:e59607. https:\/\/doi.org\/10.2196\/59607","journal-title":"JMIR Med Educ"},{"key":"6435_CR17","doi-asserted-by":"publisher","first-page":"e59505","DOI":"10.2196\/59505","volume":"26","author":"R AlSaad","year":"2024","unstructured":"AlSaad R, Aloqaily M, Guizani M et al (2024) Multimodal large language models in health care: applications, challenges, and future directions. J Med Internet Res 26:e59505. https:\/\/doi.org\/10.2196\/59505","journal-title":"J Med Internet Res"},{"key":"6435_CR18","doi-asserted-by":"publisher","unstructured":"Nam Y et al (2025) Multimodal large language models in medical imaging: current state and future directions. https:\/\/doi.org\/10.3348\/kjr.2025.0599","DOI":"10.3348\/kjr.2025.0599"},{"key":"6435_CR19","doi-asserted-by":"publisher","unstructured":"Han F et al (2025) Artificial intelligence in orthopedic surgery. https:\/\/doi.org\/10.1002\/mco2.70260","DOI":"10.1002\/mco2.70260"},{"key":"6435_CR20","doi-asserted-by":"publisher","unstructured":"Elkohail A et al (2025) Artificial intelligence in bone fracture detection: review of evidence, limitations, and clinical integration. https:\/\/doi.org\/10.7759\/cureus.97674","DOI":"10.7759\/cureus.97674"},{"key":"6435_CR21","doi-asserted-by":"publisher","unstructured":"Hsieh MY et al (2025) Impact of prompt engineering on the performance of ChatGPT in medical examinations. https:\/\/doi.org\/10.2196\/78320","DOI":"10.2196\/78320"},{"key":"6435_CR22","doi-asserted-by":"publisher","DOI":"10.1038\/s41598-024-66933-x","author":"D Patel","year":"2024","unstructured":"Patel D et al (2024) Evaluating prompt engineering on GPT-3.5\u2019s performance on USMLE-style clinical and calculation questions. Sci Rep. https:\/\/doi.org\/10.1038\/s41598-024-66933-x","journal-title":"Sci Rep"}],"container-title":["Archives of Orthopaedic and Trauma Surgery"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s00402-026-06435-9.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s00402-026-06435-9","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s00402-026-06435-9.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,8,10]],"date-time":"2026-08-10T04:00:35Z","timestamp":1786334435000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s00402-026-06435-9"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,8,10]]},"references-count":22,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2026,12]]}},"alternative-id":["6435"],"URL":"https:\/\/doi.org\/10.1007\/s00402-026-06435-9","relation":{},"ISSN":["1434-3916"],"issn-type":[{"value":"1434-3916","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,8,10]]},"assertion":[{"value":"7 April 2026","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"16 July 2026","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"10 August 2026","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"The authors declare no competing interests.","order":1,"name":"Ethics","label":"Conflict of interest","group":{"name":"EthicsHeading","label":"Declarations"}}],"article-number":"291"}}