{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,15]],"date-time":"2026-10-15T16:44:33Z","timestamp":1792082673140,"version":"4.4.0"},"reference-count":27,"publisher":"Wiley","issue":"4","license":[{"start":{"date-parts":[[2026,10,3]],"date-time":"2026-10-03T00:00:00Z","timestamp":1790985600000},"content-version":"vor","delay-in-days":2,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"},{"start":{"date-parts":[[2026,10,1]],"date-time":"2026-10-01T00:00:00Z","timestamp":1790812800000},"content-version":"tdm","delay-in-days":0,"URL":"http:\/\/doi.wiley.com\/10.1002\/tdm_license_1.1"}],"content-domain":{"domain":["esskajournals.onlinelibrary.wiley.com"],"crossmark-restriction":true},"short-container-title":["J. exp. orthop."],"published-print":{"date-parts":[[2026,10]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:sec>\n                    <jats:title>Purpose<\/jats:title>\n                    <jats:p>Large language models (LLMs) are increasingly used in clinical contexts; however, performance in complex perioperative decision\u2010making remains uncertain. Orthopaedic anaesthesia presents a demanding test case due to comorbidity burden and guideline\u2010dependent management. Whether successive LLM generations and guideline\u2010augmented prompting improve clinical alignment, comparative performance and response consistency was evaluated in this study.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Methods<\/jats:title>\n                    <jats:p>In this controlled prompting study, 34 orthopaedic anaesthesia questions spanning seven clinical subdomains were independently answered by three LLMs (GPT\u20103.5\u2010turbo, GPT\u20104o and GPT\u20105.2) and three expert anesthesiologists. GPT\u20104o and GPT\u20105.2 each generated responses with and without access to relevant clinical practice guidelines. All responses were anonymized and evaluated in blinded pairwise comparisons by an independent guideline\u2010informed LLM\u2010as\u2010a\u2010judge (GPT\u20105.2) using the same guideline material as the reference standard. The judge recorded preference, response consistency and confidence. Intra\u2010model consistency was assessed from repeated independent generations of each question.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>All LLMs were preferred over human experts in pairwise comparisons without guideline augmentation, with win rates of 67.6% (GPT\u20103.5\u2010turbo), 79.4% (GPT\u20104o) and 95.1% (GPT\u20105.2). Performance was improved by guideline augmentation, most notably for GPT\u20104o (91.2%, +11.8 percentage points), while GPT\u20105.2 approached ceiling performance (97.1%). Response consistency varied between models. GPT\u20104o showed the highest baseline consistency (80.4%), whereas GPT\u20105.2 demonstrated no fully contradictory outputs but showed greater partial variability. Guideline augmentation numerically improved GPT\u20105.2 consistency (66.7% to 80.4%) and reduced inter\u2010model differences. Directed qualitative analysis suggested that reviewer preference for later GPT generations was associated with greater completeness, explicit clinical reasoning and guideline\u2010oriented responses.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusion<\/jats:title>\n                    <jats:p>Successive LLM generations demonstrated progressively improved performance in orthopaedic anaesthesia reasoning. Guideline\u2010augmented prompting enhanced response quality, particularly for intermediate models. Guideline\u2010informed LLM\u2010as\u2010a\u2010judge evaluation appears promising for comparative assessment but requires further validation.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Level of Evidence<\/jats:title>\n                    <jats:p>NA.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.1002\/jeo2.70922","type":"journal-article","created":{"date-parts":[[2026,10,3]],"date-time":"2026-10-03T09:01:31Z","timestamp":1791018091000},"update-policy":"https:\/\/doi.org\/10.1002\/crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Guideline\u2010augmented prompting improves comparative preference and response consistency of large language model outputs for orthopaedic anaesthesia questions: A controlled prompting study"],"prefix":"10.1002","volume":"13","author":[{"ORCID":"https:\/\/orcid.org\/0009-0007-7202-4860","authenticated-orcid":false,"given":"Anita","family":"Sz\u00e9ll","sequence":"first","affiliation":[{"name":"Department of Anesthesiology and Intensive Care Medicine, Institute of Clinical Sciences Sahlgrenska Academy Gothenburg Sweden"},{"name":"Department of Anesthesiology and Intensive Care Medicine Sahlgrenska University Hospital M\u00f6lndal M\u00f6lndal Sweden"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yinan","family":"Yu","sequence":"additional","affiliation":[{"name":"Department of Computer Science and Engineering Chalmers University of Technology Gothenburg Sweden"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jacob F.","family":"Oeding","sequence":"additional","affiliation":[{"name":"Department of Orthopaedics, Institute of Clinical Science Sahlgrenska Academy Gothenburg Sweden"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Felix C.","family":"Oettl","sequence":"additional","affiliation":[{"name":"Department of Orthopaedics, Institute of Clinical Science Sahlgrenska Academy Gothenburg Sweden"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Adam","family":"Piasecki","sequence":"additional","affiliation":[{"name":"Department of Anesthesiology and Intensive Care Medicine, Institute of Clinical Sciences Sahlgrenska Academy Gothenburg Sweden"},{"name":"Department of Anesthesiology and Intensive Care Medicine Sahlgrenska University Hospital M\u00f6lndal M\u00f6lndal Sweden"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Keti","family":"Dalla","sequence":"additional","affiliation":[{"name":"Department of Anesthesiology and Intensive Care Medicine, Institute of Clinical Sciences Sahlgrenska Academy Gothenburg Sweden"},{"name":"Department of Anesthesiology and Intensive Care Medicine Sahlgrenska University Hospital Gothenburg Sweden"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Tobias","family":"Si\u00f6land","sequence":"additional","affiliation":[{"name":"Department of Anesthesiology and Intensive Care Medicine, Institute of Clinical Sciences Sahlgrenska Academy Gothenburg Sweden"},{"name":"Department of Anesthesiology and Intensive Care Medicine Sahlgrenska University Hospital M\u00f6lndal M\u00f6lndal Sweden"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Mathias","family":"H\u00e5rd af Segerstad","sequence":"additional","affiliation":[{"name":"Department of Anesthesiology and Intensive Care Medicine, Institute of Clinical Sciences Sahlgrenska Academy Gothenburg Sweden"},{"name":"Department of Anesthesiology and Intensive Care Medicine Sahlgrenska University Hospital Gothenburg Sweden"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Fredrik","family":"Olsen","sequence":"additional","affiliation":[{"name":"Department of Anesthesiology and Intensive Care Medicine, Institute of Clinical Sciences Sahlgrenska Academy Gothenburg Sweden"},{"name":"Department of Anesthesiology and Intensive Care Medicine Sahlgrenska University Hospital Gothenburg Sweden"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Peter","family":"Larsson","sequence":"additional","affiliation":[{"name":"Department of Anesthesiology and Intensive Care Medicine, Institute of Clinical Sciences Sahlgrenska Academy Gothenburg Sweden"},{"name":"Department of Anesthesiology and Intensive Care Medicine Sahlgrenska University Hospital Gothenburg Sweden"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Stefano","family":"Zaffagnini","sequence":"additional","affiliation":[{"name":"Clinica Ortopedica e Traumatologica II IRCCS Istituto Ortopedico Rizzoli Bologna Italy"},{"name":"Dipartimento di Scienze Biomediche eNeuromotorie (DIBINEM) University of Bologna Bologna Italy"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Kristian","family":"Samuelsson","sequence":"additional","affiliation":[{"name":"Department of Orthopaedics, Institute of Clinical Science Sahlgrenska Academy Gothenburg Sweden"},{"name":"Department of Orthopaedics Sahlgrenska University Hospital M\u00f6lndal M\u00f6lndal Sweden"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Fredrik","family":"Hessulf","sequence":"additional","affiliation":[{"name":"Department of Anesthesiology and Intensive Care Medicine, Institute of Clinical Sciences Sahlgrenska Academy Gothenburg Sweden"},{"name":"Department of Anesthesiology and Intensive Care Medicine Sahlgrenska University Hospital M\u00f6lndal M\u00f6lndal Sweden"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"311","published-online":{"date-parts":[[2026,10,3]]},"reference":[{"key":"e_1_2_12_2_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10439-023-03325-8"},{"key":"e_1_2_12_3_1","doi-asserted-by":"publisher","DOI":"10.1001\/jamainternmed.2023.1838"},{"key":"e_1_2_12_4_1","doi-asserted-by":"publisher","DOI":"10.1177\/08850666241267871"},{"key":"e_1_2_12_5_1","doi-asserted-by":"publisher","DOI":"10.1080\/17843286.2026.2613903"},{"key":"e_1_2_12_6_1","doi-asserted-by":"publisher","DOI":"10.1186\/s12871-024-02882-2"},{"key":"e_1_2_12_7_1","doi-asserted-by":"publisher","DOI":"10.1186\/s40634-023-00700-1"},{"key":"e_1_2_12_8_1","unstructured":"ChenGH ChenS LiuZ JiangF WangB. Humans or LLMs as the judge? A study on judgement biases. arXiv; 2024:arXiv:2402.10669. Available from:https:\/\/doi.org\/10.48550\/arXiv.2402.10669"},{"key":"e_1_2_12_9_1","doi-asserted-by":"publisher","DOI":"10.1186\/s13741-024-00469-6"},{"key":"e_1_2_12_10_1","unstructured":"ChungP SwaminathanA GoodellAJ KimY ReinckeSM HanL et al. VeriFact: verifying facts in LLM\u2010generated clinical text with electronic health records. arXiv; 2025:arXiv:2501.16672. Available from:https:\/\/arxiv.org\/abs\/2501.16672."},{"key":"e_1_2_12_11_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jclinane.2026.112164"},{"key":"e_1_2_12_12_1","doi-asserted-by":"crossref","unstructured":"CroxfordE GaoY FirstE PellegrinoN SchnierM CaskeyJ et al. Automating evaluation of AI text generation in healthcare with a large language model (LLM)\u2010as\u2010a\u2010judge. medRxiv; 2025:2025.04.22.25326219. Available from:https:\/\/doi.org\/10.1101\/2025.04.22.25326219","DOI":"10.1101\/2025.04.22.25326219"},{"key":"e_1_2_12_13_1","doi-asserted-by":"publisher","DOI":"10.1038\/s41746-024-01356-6"},{"key":"e_1_2_12_14_1","doi-asserted-by":"publisher","DOI":"10.2196\/45312"},{"key":"e_1_2_12_15_1","doi-asserted-by":"publisher","DOI":"10.1177\/1049732305276687"},{"key":"e_1_2_12_16_1","doi-asserted-by":"publisher","DOI":"10.1007\/s00167-023-07529-2"},{"key":"e_1_2_12_17_1","doi-asserted-by":"publisher","DOI":"10.1002\/jeo2.12047"},{"key":"e_1_2_12_18_1","unstructured":"LiuP YuanW FuJ JiangZ HayashiH NeubigG. Pre\u2010train prompt and predict: a systematic survey of prompting methods in natural language processing. arXiv; 2021:arXiv:2107.13586. Available from:https:\/\/doi.org\/10.48550\/arXiv.2107.13586"},{"key":"e_1_2_12_19_1","doi-asserted-by":"publisher","DOI":"10.1002\/jeo2.70273"},{"key":"e_1_2_12_20_1","doi-asserted-by":"publisher","DOI":"10.1002\/jeo2.70135"},{"key":"e_1_2_12_21_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.arthro.2024.08.036"},{"key":"e_1_2_12_22_1","doi-asserted-by":"publisher","DOI":"10.1245\/s10434-023-13642-w"},{"key":"e_1_2_12_23_1","doi-asserted-by":"publisher","DOI":"10.1056\/AIoa2300015"},{"key":"e_1_2_12_24_1","doi-asserted-by":"publisher","DOI":"10.1056\/AIe2501329"},{"key":"e_1_2_12_25_1","doi-asserted-by":"publisher","DOI":"10.3390\/diagnostics16010003"},{"key":"e_1_2_12_26_1","doi-asserted-by":"publisher","DOI":"10.1038\/s41746-026-02992-w"},{"key":"e_1_2_12_27_1","doi-asserted-by":"crossref","unstructured":"ZhengL ChiangWL ShengY ZhuangS WuZ ZhuangY et al. Judging LLM\u2010as\u2010a\u2010judge with MT\u2010bench and chatbot arena. arXiv: 2023:arXiv:2306.05685. Available from:https:\/\/doi.org\/10.48550\/arXiv.2306.05685","DOI":"10.52202\/075280-2020"},{"key":"e_1_2_12_28_1","doi-asserted-by":"publisher","DOI":"10.1038\/s41746-025-02208-7"}],"container-title":["Journal of Experimental Orthopaedics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/esskajournals.onlinelibrary.wiley.com\/doi\/pdf\/10.1002\/jeo2.70922","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/esskajournals.onlinelibrary.wiley.com\/doi\/full-xml\/10.1002\/jeo2.70922","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/esskajournals.onlinelibrary.wiley.com\/doi\/pdf\/10.1002\/jeo2.70922","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,10,15]],"date-time":"2026-10-15T15:45:31Z","timestamp":1792079131000},"score":1,"resource":{"primary":{"URL":"https:\/\/esskajournals.onlinelibrary.wiley.com\/doi\/10.1002\/jeo2.70922"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,10]]},"references-count":27,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2026,10]]}},"alternative-id":["10.1002\/jeo2.70922"],"URL":"https:\/\/doi.org\/10.1002\/jeo2.70922","archive":["Portico"],"relation":{},"ISSN":["2197-1153","2197-1153"],"issn-type":[{"value":"2197-1153","type":"print"},{"value":"2197-1153","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,10]]},"assertion":[{"value":"2026-05-24","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2026-09-09","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2026-10-03","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}],"article-number":"e70922"}}