{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,28]],"date-time":"2026-08-28T03:07:30Z","timestamp":1787886450453,"version":"build-2784847793"},"reference-count":31,"publisher":"SAGE Publications","issue":"6","license":[{"start":{"date-parts":[[2025,4,1]],"date-time":"2025-04-01T00:00:00Z","timestamp":1743465600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/journals.sagepub.com\/page\/policies\/text-and-data-mining-license"}],"content-domain":{"domain":["journals.sagepub.com"],"crossmark-restriction":true},"short-container-title":["Sports Health: A Multidisciplinary Approach"],"published-print":{"date-parts":[[2025,11]]},"abstract":"<jats:sec>\n                    <jats:title>Background:<\/jats:title>\n                    <jats:p>Large language model (LLM)-based artificial intelligence (AI) chatbots, such as ChatGPT and Gemini, have become widespread sources of information. Few studies have evaluated LLM responses to questions about orthopaedic conditions, especially osteochondritis dissecans (OCD).<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Hypothesis:<\/jats:title>\n                    <jats:p>ChatGPT and Gemini will generate accurate responses that align with American Academy of Orthopaedic Surgeons (AAOS) clinical practice guidelines.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Study Design:<\/jats:title>\n                    <jats:p>Cohort study.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Level of Evidence:<\/jats:title>\n                    <jats:p>Level 2.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Methods:<\/jats:title>\n                    <jats:p>LLM prompts were created based on AAOS clinical guidelines on OCD diagnosis and treatment, and responses from ChatGPT and Gemini were collected. Seven fellowship-trained orthopaedic surgeons evaluated LLM responses on a 5-point Likert scale, based on 6 categories: relevance, accuracy, clarity, completeness, evidence-based, and consistency.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results:<\/jats:title>\n                    <jats:p>ChatGPT and Gemini exhibited strong performance across all criteria. ChatGPT mean scores were highest for clarity (4.771 \u00b1 0.141 [mean \u00b1 SD]). Gemini scored highest for relevance and accuracy (4.286 \u00b1 0.296, 4.286 \u00b1 0.273). For both LLMs, the lowest scores were for evidence-based responses (ChatGPT, 3.857 \u00b1 0.352; Gemini, 3.743 \u00b1 0.353). For all other categories, ChatGPT mean scores were higher than Gemini scores. The consistency of responses between the 2 LLMs was rated at an overall mean of 3.486 \u00b1 0.371. Inter-rater reliability ranged from 0.4 to 0.67 (mean, 0.59) and was highest (0.67) in the accuracy category and lowest (0.4) in the consistency category.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusion:<\/jats:title>\n                    <jats:p>LLM performance emphasizes the potential for gathering clinically relevant and accurate answers to questions regarding the diagnosis and treatment of OCD and suggests that ChatGPT may be a better model for this purpose than the Gemini model. Further evaluation of LLM information regarding other orthopaedic procedures and conditions may be necessary before LLMs can be recommended as an accurate source of orthopaedic information.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Clinical Relevance:<\/jats:title>\n                    <jats:p>Little is known about the ability of AI to provide answers regarding OCD.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.1177\/19417381251326549","type":"journal-article","created":{"date-parts":[[2025,4,2]],"date-time":"2025-04-02T02:29:50Z","timestamp":1743560990000},"page":"1340-1346","update-policy":"https:\/\/doi.org\/10.1177\/sage-journals-update-policy","source":"Crossref","is-referenced-by-count":2,"title":["Performance of Artificial Intelligence in Addressing Questions Regarding Management of Osteochondritis Dissecans"],"prefix":"10.1177","volume":"17","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-4888-6061","authenticated-orcid":false,"given":"John D.","family":"Milner","sequence":"first","affiliation":[{"name":"Department of Orthopaedic Surgery, Brown University, Warren Alpert Medical School, Providence, Rhode Island"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3561-6844","authenticated-orcid":false,"given":"Matthew S.","family":"Quinn","sequence":"additional","affiliation":[{"name":"Department of Orthopaedic Surgery, Brown University, Warren Alpert Medical School, Providence, Rhode Island"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Phillip","family":"Schmitt","sequence":"additional","affiliation":[{"name":"Department of Orthopaedic Surgery, Brown University, Warren Alpert Medical School, Providence, Rhode Island"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Rigel P.","family":"Hall","sequence":"additional","affiliation":[{"name":"Creighton University School of Medicine, Phoenix, Arizona"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Steven","family":"Bokshan","sequence":"additional","affiliation":[{"name":"Cone Health Orthopedics, Greensboro, North Carolina"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Logan","family":"Petit","sequence":"additional","affiliation":[{"name":"Connecticut Orthopaedics, Hamden, Connecticut"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ryan","family":"O\u2019Donnell","sequence":"additional","affiliation":[{"name":"Department of Orthopedic Surgery, St Luke\u2019s University Health Network, Bethlehem, Pennsylvania"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Stephen E.","family":"Marcaccio","sequence":"additional","affiliation":[{"name":"Department of Orthopaedic Surgery, Brown University, Warren Alpert Medical School, Providence, Rhode Island"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8518-5371","authenticated-orcid":false,"given":"Steven F.","family":"DeFroda","sequence":"additional","affiliation":[{"name":"Department of Orthopedic Surgery, Missouri Orthopedic Institute, University of Missouri, Columbia, Missouri"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ramin R.","family":"Tabaddor","sequence":"additional","affiliation":[{"name":"Department of Orthopaedic Surgery, Brown University, Warren Alpert Medical School, Providence, Rhode Island"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9972-0096","authenticated-orcid":false,"given":"Brett D.","family":"Owens","sequence":"additional","affiliation":[{"name":"Department of Orthopaedic Surgery, Brown University, Warren Alpert Medical School, Providence, Rhode Island"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"179","published-online":{"date-parts":[[2025,4]]},"reference":[{"key":"e_1_3_2_2_2","doi-asserted-by":"publisher","DOI":"10.1097\/BPO.0000000000002681"},{"key":"e_1_3_2_3_2","doi-asserted-by":"publisher","DOI":"10.1186\/s12909-023-04698-z"},{"key":"e_1_3_2_4_2","doi-asserted-by":"publisher","DOI":"10.1097\/BPO.0000000000002695"},{"issue":"5","key":"e_1_3_2_5_2","article-title":"High rates of fabricated and inaccurate references in ChatGPT-generated medical content","volume":"15","author":"Bhattacharyya M","year":"2023","unstructured":"Bhattacharyya M, Miller VM, Bhattacharyya D, Miller LE. High rates of fabricated and inaccurate references in ChatGPT-generated medical content. Cureus. 2023;15(5):e39238.","journal-title":"Cureus"},{"key":"e_1_3_2_6_2","doi-asserted-by":"publisher","DOI":"10.1007\/s11420-017-9568-2"},{"key":"e_1_3_2_7_2","doi-asserted-by":"crossref","unstructured":"ChatGPT. https:\/\/chat.openai.com. Accessed March 29 2024.","DOI":"10.1145\/3704289.3704301"},{"key":"e_1_3_2_8_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jhsa.2023.10.019"},{"key":"e_1_3_2_9_2","doi-asserted-by":"publisher","DOI":"10.3928\/01913913-20240409-01"},{"key":"e_1_3_2_10_2","doi-asserted-by":"publisher","DOI":"10.3390\/jcm12206655"},{"issue":"2","key":"e_1_3_2_11_2","article-title":"Readability of patient education materials on the American Association for Surgery of Trauma website","volume":"3","author":"Eltorai AEM","year":"2014","unstructured":"Eltorai AEM, Ghanian S, Adams CAJr, Born CT, Daniels AH. Readability of patient education materials on the American Association for Surgery of Trauma website. Arch Trauma Res. 2014;3(2):e18161.","journal-title":"Arch Trauma Res"},{"key":"e_1_3_2_12_2","unstructured":"Gemini. Gemini - supercharge your creativity and productivity. https:\/\/gemini.google.com. Accessed March 29 2024."},{"issue":"12","key":"e_1_3_2_13_2","article-title":"Google Bard and ChatGPT in orthopedics: which is the better doctor in sports medicine and pediatric orthopedics? The role of AI in patient education","volume":"14","author":"Giorgino R","year":"2024","unstructured":"Giorgino R, Alessandri-Bonetti M, Del Re M, Verdoni F, Peretti GM, Mangiavini L. Google Bard and ChatGPT in orthopedics: which is the better doctor in sports medicine and pediatric orthopedics? The role of AI in patient education. Diagn Basel Switz. 2024;14(12):1253.","journal-title":"Diagn Basel Switz"},{"key":"e_1_3_2_14_2","doi-asserted-by":"publisher","DOI":"10.3390\/healthcare12111083"},{"key":"e_1_3_2_15_2","doi-asserted-by":"publisher","DOI":"10.3390\/app13137623"},{"key":"e_1_3_2_16_2","doi-asserted-by":"publisher","DOI":"10.1001\/jamanetworkopen.2022.10309"},{"issue":"4","key":"e_1_3_2_17_2","first-page":"498","article-title":"Understanding osteochondritis dissecans: a narrative review of the disease commonly affecting children and adolescents","volume":"11","author":"Konarski W","year":"2024","unstructured":"Konarski W, Pobo\u017cy T, Konarska K, Derczy\u0144ski M, Kotela I. Understanding osteochondritis dissecans: a narrative review of the disease commonly affecting children and adolescents. Children (Basel). 2024;11(4):498.","journal-title":"Children (Basel)"},{"issue":"8","key":"e_1_3_2_18_2","article-title":"Artificial hallucinations by Google Bard: think before you leap","volume":"15","author":"Kumar M","year":"2023","unstructured":"Kumar M, Mani UA, Tripathi P, Saalim M, Roy S. Artificial hallucinations by Google Bard: think before you leap. Cureus. 2023;15(8):e43313.","journal-title":"Cureus"},{"key":"e_1_3_2_19_2","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pdig.0000198"},{"issue":"3","key":"e_1_3_2_20_2","article-title":"Generative artificial intelligence performs at a second-year orthopedic resident level","volume":"16","author":"Lum ZC","year":"2024","unstructured":"Lum ZC, Collins DP, Dennison S, et al. Generative artificial intelligence performs at a second-year orthopedic resident level. Cureus. 2024;16(3):e56104.","journal-title":"Cureus"},{"key":"e_1_3_2_21_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.arth.2024.02.023"},{"key":"e_1_3_2_22_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.psychres.2023.115334"},{"key":"e_1_3_2_23_2","doi-asserted-by":"publisher","DOI":"10.11613\/BM.2012.031"},{"key":"e_1_3_2_24_2","doi-asserted-by":"publisher","DOI":"10.2106\/JBJS.23.00209"},{"key":"e_1_3_2_25_2","doi-asserted-by":"publisher","DOI":"10.1177\/2325967123S00071"},{"key":"e_1_3_2_26_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.arthro.2024.09.020"},{"key":"e_1_3_2_27_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.iotcps.2023.04.003"},{"key":"e_1_3_2_28_2","unstructured":"Rothwell J. Assessing the economic gains of eradicating illiteracy nationally and regionally in the United States. https:\/\/www.barbarabush.org\/wp-content\/uploads\/2020\/09\/BBFoundation_GainsFromEradicatingIlliteracy_9_8.pdf"},{"key":"e_1_3_2_29_2","doi-asserted-by":"publisher","DOI":"10.1177\/03635465221142280"},{"issue":"6","key":"e_1_3_2_30_2","article-title":"Inadequate performance of ChatGPT on orthopedic board-style written exams","volume":"16","author":"Sparks CA","year":"2024","unstructured":"Sparks CA, Kraeutler MJ, Chester GA, et al. Inadequate performance of ChatGPT on orthopedic board-style written exams. Cureus. 2024;16(6):e62643.","journal-title":"Cureus"},{"key":"e_1_3_2_31_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.arth.2024.02.005"},{"key":"e_1_3_2_32_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.arth.2024.01.029"}],"container-title":["Sports Health: A Multidisciplinary Approach"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/19417381251326549","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/full-xml\/10.1177\/19417381251326549","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/19417381251326549","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,4,30]],"date-time":"2026-04-30T22:49:03Z","timestamp":1777589343000},"score":1,"resource":{"primary":{"URL":"https:\/\/journals.sagepub.com\/doi\/10.1177\/19417381251326549"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,4]]},"references-count":31,"journal-issue":{"issue":"6","published-print":{"date-parts":[[2025,11]]}},"alternative-id":["10.1177\/19417381251326549"],"URL":"https:\/\/doi.org\/10.1177\/19417381251326549","relation":{},"ISSN":["1941-7381","1941-0921"],"issn-type":[{"value":"1941-7381","type":"print"},{"value":"1941-0921","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,4]]}}}