{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,25]],"date-time":"2026-08-25T15:19:29Z","timestamp":1787671169859,"version":"build-2736575974"},"reference-count":12,"publisher":"Ovid Technologies (Wolters Kluwer Health)","issue":"15","content-domain":{"domain":["lww.com","ovid.com"],"crossmark-restriction":true},"short-container-title":["J Am Acad Orthop Surg","JAAOS"],"published-print":{"date-parts":[[2026,8,1]]},"abstract":"<jats:sec>\n                    <jats:title>Background:<\/jats:title>\n                    <jats:p>This study sought to assess large language models' (LLM) ability to generate correct ICD-10 and CPT codes using clinical documentation, and to determine whether note authorship, prompt design, or diagnosis\/procedure type affect LLM performance. We hypothesized that LLMs can code hand surgery clinic and surgical notes with greater than 80% accuracy.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Methods:<\/jats:title>\n                    <jats:p>\n                      Ninety patients evenly distributed across three orthopaedic hand surgeons and procedure types (cubital tunnel, carpal tunnel, and trigger finger release) were identified. Clinic and surgical notes were deidentified, and correct ICD-10 diagnosis and CPT procedure codes were recorded. \u201cZero-shot,\u201d \u201cone-shot,\u201d \u201cmultishot,\u201d and \u201cchain-of-thought\u201d prompts instructed LLMs to assign ICD-10 codes and CPT codes based on note content. Each prompt was posed to Chat GPT 3.5, Chat GPT 4.0, and Gemini. Rates of coding correctness were calculated across attendings, diagnosis\/procedure, prompt type, and LLM. Chi-square analysis determined statistical significance for these comparisons (\n                      <jats:italic toggle=\"yes\">P<\/jats:italic>\n                      &lt; 0.05).\n                    <\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results:<\/jats:title>\n                    <jats:p>\n                      No differences in LLM coding performance were observed between note authors (\n                      <jats:italic toggle=\"yes\">P<\/jats:italic>\n                      = 0.09 ICD-10,\n                      <jats:italic toggle=\"yes\">P<\/jats:italic>\n                      = 0.48 CPT) or prompt types (\n                      <jats:italic toggle=\"yes\">P<\/jats:italic>\n                      = 0.27 ICD-10,\n                      <jats:italic toggle=\"yes\">P<\/jats:italic>\n                      = 0.62 CPT). Chat GPT 3.5 provided less accurate ICD-10 codes than Chat GPT 4.0 or Gemini (\n                      <jats:italic toggle=\"yes\">P<\/jats:italic>\n                      &lt; 0.0001). All LLMs better predicted CPT codes (91.5% correct) than ICD-10 codes (23.9% correct). The most common error was incorrect or omitted ICD-10 laterality. Prompts updated to emphasize ICD-10 laterality demonstrated improved accuracy (40%).\n                    <\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Discussion:<\/jats:title>\n                    <jats:p>Variation in note content and writing style did not markedly affect LLM performance. Public-facing LLMs require additional optimization to interpret clinical documentation for coding purposes and are not ready for independent use.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.5435\/jaaos-d-25-01463","type":"journal-article","created":{"date-parts":[[2026,5,22]],"date-time":"2026-05-22T13:00:57Z","timestamp":1779454857000},"page":"e2164-e2172","update-policy":"https:\/\/doi.org\/10.1097\/lww.0000000000001000","source":"Crossref","is-referenced-by-count":0,"title":["Assessing Large Language Models for Clinical Coding in Hand Surgery: Effect of Note Authorship, Prompt Design, and Diagnosis\/Procedure Type"],"prefix":"10.5435","volume":"34","author":[{"ORCID":"https:\/\/orcid.org\/0009-0006-5051-8192","authenticated-orcid":false,"given":"Avery M.","family":"Schroeder","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Carlye B.","family":"Goldenberg","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Mubinah I.","family":"Khaleel","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Julia A. V.","family":"Nuelle","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Benjamin J.","family":"Kirby","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1711-0762","authenticated-orcid":false,"given":"Daniel A.","family":"London","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"276","published-online":{"date-parts":[[2026,5,22]]},"reference":[{"key":"R2-20260729","doi-asserted-by":"crossref","first-page":"1169595","DOI":"10.3389\/frai.2023.1169595","article-title":"ChatGPT in medicine: An overview of its applications, advantages, limitations, future prospects, and ethical considerations","volume":"6","author":"Dave","year":"2023","journal-title":"Front Artif Intell"},{"key":"R3-20260729","doi-asserted-by":"crossref","first-page":"1","DOI":"10.13107\/jocr.2024.v14.i02.4194","article-title":"The rise of artificial intelligence: Implications in orthopedic surgery","volume":"14","author":"Paul","year":"2024","journal-title":"J Orthop Case Rep"},{"key":"R4-20260729","doi-asserted-by":"crossref","first-page":"20159","DOI":"10.1038\/s41598-023-47500-2","article-title":"A pilot study on the efficacy of GPT-4 in providing orthopedic treatment recommendations from MRI reports","volume":"13","author":"Truhn","year":"2023","journal-title":"Sci Rep"},{"key":"R5-20260729","doi-asserted-by":"crossref","first-page":"1096","DOI":"10.1097\/SLA.0000000000000851","article-title":"A study of clinical coding accuracy in surgery: Implications for the use of administrative big data for outcomes management","volume":"261","author":"Nouraei","year":"2015","journal-title":"Ann Surg"},{"key":"R6-20260729","doi-asserted-by":"crossref","first-page":"105","DOI":"10.1097\/BLO.0b013e31803372b8","article-title":"Coding basics for orthopaedic surgeons","volume":"457","author":"Filler","year":"2007","journal-title":"Clin Orthop Relat Res"},{"key":"R7-20260729","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1056\/AIdbp2300040","article-title":"Large language models are poor medical coders \u2013 Benchmarking of medical code querying","volume":"1","author":"Soroush","year":"2024","journal-title":"NEJM AI"},{"key":"R8-20260729","doi-asserted-by":"crossref","first-page":"1946","DOI":"10.1177\/21925682211062831","article-title":"Can natural language processing and artificial intelligence automate the generation of billing codes from operative note dictations?","volume":"13","author":"Kim","year":"2023","journal-title":"Glob Spine J"},{"key":"R9-20260729","first-page":"15589447241295328","article-title":"Chatbot demonstrates moderate interrater reliability in billing for hand surgery clinic encounters","volume":"16","author":"Latario","year":"2024","journal-title":"Hand (NY)"},{"key":"R10-20260729","doi-asserted-by":"crossref","first-page":"181","DOI":"10.1016\/j.jhsg.2024.11.013","article-title":"Artificial intelligence in surgical coding: Evaluating large language models for current procedural terminology accuracy in hand surgery","volume":"7","author":"Isch","year":"2025","journal-title":"J Hand Surg Glob Online"},{"key":"R12-20260729","doi-asserted-by":"crossref","first-page":"24824","DOI":"10.52202\/068431-1800","article-title":"Chain-of-thought prompting elicits reasoning in large language models","volume":"35","author":"Wei","year":"2022","journal-title":"Adv Neural Inf Process Syst"},{"key":"R13-20260729","doi-asserted-by":"crossref","first-page":"445","DOI":"10.3390\/medicina60030445","article-title":"Integrating retrieval-augmented generation with large language models in nephrology: Advancing practical applications","volume":"60","author":"Miao","year":"2024","journal-title":"Medicina"},{"key":"R14-20260729","doi-asserted-by":"crossref","first-page":"e50638","DOI":"10.2196\/50638","article-title":"Prompt engineering as an important emerging skill for medical professionals: Tutorial","volume":"25","author":"Mesk\u00f3","year":"2023","journal-title":"J Med Internet Res"}],"container-title":["Journal of the American Academy of Orthopaedic Surgeons"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.ovid.com\/10.5435\/JAAOS-D-25-01463","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,8,25]],"date-time":"2026-08-25T14:39:56Z","timestamp":1787668796000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.ovid.com\/10.5435\/JAAOS-D-25-01463"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,5,22]]},"references-count":12,"journal-issue":{"issue":"15","published-print":{"date-parts":[[2026]]}},"URL":"https:\/\/doi.org\/10.5435\/jaaos-d-25-01463","relation":{},"ISSN":["1067-151X","1940-5480"],"issn-type":[{"value":"1067-151X","type":"print"},{"value":"1940-5480","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,5,22]]},"assertion":[{"value":"2025-11-24","name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2026-03-30","name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}}]}}