{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,16]],"date-time":"2026-10-16T15:44:36Z","timestamp":1792165476527,"version":"4.4.0"},"reference-count":13,"publisher":"Ovid Technologies (Wolters Kluwer Health)","content-domain":{"domain":["lww.com","ovid.com"],"crossmark-restriction":true},"short-container-title":[],"abstract":"\n Background:<\/jats:title>\n The use of large language models (LLMs) is increasingly common. However, LLMs may exhibit sycophancy, echoing users\u2019 beliefs while avoiding contradiction. In the present study, we describe sycophancy in general-purpose LLMs when applied to orthopaedic contexts.<\/jats:p>\n <\/jats:sec>\n \n Methods:<\/jats:title>\n We investigated sycophancy in 2 general-purpose LLMs. We evaluated performance on 3 tasks: (1) accuracy on benchmark answering: LLMs were tested on validated benchmark orthopaedic questions, with correct and incorrect cues, and the change in accuracy and sycophancy error rate were determined; (2) user belief agreement: LLMs were provided with ambiguous statements and a user belief, and LLM agreement, contradiction, and uncertainty were described; and (3) false information detection: false information was placed within a task prompt to measure noncontradiction and propagation rates.<\/jats:p>\n <\/jats:sec>\n \n Results:<\/jats:title>\n Baseline factual accuracy on benchmark questioning was 78%, decreasing with correct hints (71%) (p = 0.49). With incorrect hints, LLM accuracy declined significantly (48%) (p < 0.001), with a sycophancy error rate of 52%. Presented with user beliefs about an indefinite, controversial statement, models echoed user beliefs in 56%, expressed uncertainty in 12%, and contradicted users in 32% of statements. In noncontradiction tasks, models perpetuated incorrect attributions 99% of the time yet reliably corrected statistical distortions 97% of the time.<\/jats:p>\n <\/jats:sec>\n \n Conclusions:<\/jats:title>\n Although popular general-purpose LLMs have useful orthopaedic applications, they exhibit sycophancy, with a tendency toward agreement and without recognition of ambiguity. This is a key weakness to be addressed. Findings should be interpreted cautiously given the variability in model design, prompting, and models evaluated.<\/jats:p>\n <\/jats:sec>\n \n Clinical Relevance:<\/jats:title>\n The tendency of general-purpose LLMs to agree without recognizing clinical ambiguity may limit their reliability in orthopaedic applications.<\/jats:p>\n <\/jats:sec>","DOI":"10.2106\/jbjs.25.01576","type":"journal-article","created":{"date-parts":[[2026,5,21]],"date-time":"2026-05-21T18:01:00Z","timestamp":1779386460000},"update-policy":"https:\/\/doi.org\/10.1097\/lww.0000000000001000","source":"Crossref","is-referenced-by-count":1,"title":["Current Artificial Intelligence Large Language Models Exhibit Sycophantic Behavior in Orthopaedic Contexts"],"prefix":"10.2106","author":[{"ORCID":"https:\/\/orcid.org\/0009-0001-6309-6025","authenticated-orcid":false,"given":"Arthur J.","family":"Perry","sequence":"first","affiliation":[{"name":"Department of Orthopedic Surgery, NYU Langone Orthopedic Hospital, NYU Langone Health, NYU Grossman School of Medicine, New York, NY"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0575-2482","authenticated-orcid":false,"given":"Swara","family":"Kalva","sequence":"additional","affiliation":[{"name":"Department of Orthopedic Surgery, NYU Langone Orthopedic Hospital, NYU Langone Health, NYU Grossman School of Medicine, New York, NY"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0002-4679-6828","authenticated-orcid":false,"given":"Dario","family":"Fucich","sequence":"additional","affiliation":[{"name":"Department of Orthopedic Surgery, NYU Langone Orthopedic Hospital, NYU Langone Health, NYU Grossman School of Medicine, New York, NY"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1742-1584","authenticated-orcid":false,"given":"Srikar","family":"Muppidi","sequence":"additional","affiliation":[{"name":"Department of Orthopedic Surgery, NYU Langone Orthopedic Hospital, NYU Langone Health, NYU Grossman School of Medicine, New York, NY"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0005-5958-2417","authenticated-orcid":false,"given":"Manan","family":"Aggarwal","sequence":"additional","affiliation":[{"name":"Google Search AI, Google, Mountain View, California"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6294-2771","authenticated-orcid":false,"given":"Mandeep S.","family":"Virk","sequence":"additional","affiliation":[{"name":"Department of Orthopedic Surgery, NYU Langone Orthopedic Hospital, NYU Langone Health, NYU Grossman School of Medicine, New York, NY"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3923-8660","authenticated-orcid":false,"given":"Joseph D.","family":"Zuckerman","sequence":"additional","affiliation":[{"name":"Department of Orthopedic Surgery, NYU Langone Orthopedic Hospital, NYU Langone Health, NYU Grossman School of Medicine, New York, NY"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9950-5273","authenticated-orcid":false,"given":"Jie J.","family":"Yao","sequence":"additional","affiliation":[{"name":"Department of Orthopedic Surgery, NYU Langone Orthopedic Hospital, NYU Langone Health, NYU Grossman School of Medicine, New York, NY"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"276","published-online":{"date-parts":[[2026,5,21]]},"reference":[{"issue":"9","key":"bib1-20260521","doi-asserted-by":"crossref","first-page":"830","DOI":"10.2106\/JBJS.19.01128","article-title":"Artificial intelligence and orthopaedics: an introduction for clinicians","volume":"102","author":"Myers","year":"2020","journal-title":"J Bone Joint Surg Am."},{"issue":"15","key":"bib2-20260521","doi-asserted-by":"crossref","first-page":"1411","DOI":"10.2106\/JBJS.23.01417","article-title":"Large language models in orthopaedics: definitions, uses, and limitations","volume":"106","author":"Yao","year":"2024","journal-title":"J Bone Joint Surg Am."},{"key":"bib3-20260521","first-page":"11679","article-title":"On the Impact of Fine-Tuning on Chain-of-Thought Reasoning","volume":"1","author":"Lobo","year":"2025","journal-title":"Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies;Vol."},{"issue":"7","key":"bib7-20260521","doi-asserted-by":"crossref","first-page":"1633","DOI":"10.1016\/j.jse.2024.01.033","article-title":"Artificial intelligence in shoulder and elbow surgery: overview of current and future applications","volume":"33","author":"Levin","year":"2024","journal-title":"J Shoulder Elbow Surg."},{"issue":"14","key":"bib8-20260521","doi-asserted-by":"crossref","first-page":"656","DOI":"10.5435\/JAAOS-D-23-00877","article-title":"Evaluating ChatGPT\u2019s ability to answer common patient questions regarding hip fracture","volume":"32","author":"Wrenn","year":"2024","journal-title":"J Am Acad Orthop Surg."},{"issue":"6","key":"bib9-20260521","doi-asserted-by":"crossref","first-page":"2534","DOI":"10.1016\/j.jseint.2023.07.018","article-title":"Breaking barriers: can ChatGPT compete with a shoulder and elbow specialist in diagnosis and management?","volume":"7","author":"Daher","year":"2023","journal-title":"JSES Int."},{"issue":"3","key":"bib10-20260521","first-page":"e23.00056","article-title":"Evaluating ChatGPT performance on the orthopaedic in-training examination","volume":"8","author":"Kung","year":"2023","journal-title":"JB JS Open Access."},{"key":"bib13-20260521","article-title":"Calibrate Before Use: Improving Few-Shot Performance of Language Models","author":"Zhao","year":"2021","journal-title":"International Conference on Machine Learning"},{"issue":"4","key":"bib14-20260521","doi-asserted-by":"crossref","first-page":"368","DOI":"10.1177\/1758573221989669","article-title":"The fifty most cited publications in shoulder arthroplasty research","volume":"14","author":"Cohn","year":"2022","journal-title":"Shoulder Elbow."},{"key":"bib15-20260521","doi-asserted-by":"crossref","first-page":"132","DOI":"10.1016\/j.jor.2019.11.039","article-title":"Top 100 most cited articles in orthopaedic surgery: an update","volume":"19","author":"Lum","year":"2019","journal-title":"J Orthop."},{"issue":"1","key":"bib17-20260521","doi-asserted-by":"crossref","first-page":"605","DOI":"10.1038\/s41746-025-02008-z","article-title":"When helpfulness backfires: LLMs and the risk of false medical information due to sycophantic behavior","volume":"8","author":"Chen","year":"2025","journal-title":"NPJ Digit Med."},{"key":"bib18-20260521","doi-asserted-by":"crossref","first-page":"1780","DOI":"10.1038\/s42256-025-01113-8","article-title":"Language models cannot reliably distinguish belief from knowledge and fact","volume":"7","author":"Suzgun","year":"2025","journal-title":"Nature Machine Intelligence."},{"issue":"2","key":"bib20-20260521","doi-asserted-by":"crossref","first-page":"102","DOI":"10.33545\/27076571.2022.v3.i2a.126","article-title":"Explaining black-box behavior in large language models","volume":"3","author":"Manche","year":"2022","journal-title":"International Journal of Computing and Artificial Intelligence."}],"container-title":["Journal of Bone and Joint Surgery"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.ovid.com\/10.2106\/JBJS.25.01576","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,8,26]],"date-time":"2026-08-26T13:45:22Z","timestamp":1787751922000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.ovid.com\/10.2106\/JBJS.25.01576"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,5,21]]},"references-count":13,"URL":"https:\/\/doi.org\/10.2106\/jbjs.25.01576","relation":{},"ISSN":["0021-9355","1535-1386"],"issn-type":[{"value":"0021-9355","type":"print"},{"value":"1535-1386","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,5,21]]}}}