{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,22]],"date-time":"2026-09-22T00:12:28Z","timestamp":1790035948251,"version":"4.0.1"},"reference-count":18,"publisher":"SAGE Publications","issue":"2","license":[{"start":{"date-parts":[[2026,4,16]],"date-time":"2026-04-16T00:00:00Z","timestamp":1776297600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/journals.sagepub.com\/page\/policies\/text-and-data-mining-license"}],"content-domain":{"domain":["journals.sagepub.com"],"crossmark-restriction":true},"short-container-title":["Scott Med J"],"published-print":{"date-parts":[[2026,5]]},"abstract":"\n Objectives<\/jats:title>\n Accurate triage of lumbar spine magnetic resonance imaging (MRI) referrals for sciatica is important for patient assessment, diagnosis and surgical planning. This study evaluates the accuracy and speed of large language models (LLMs) in automatically vetting lumbar spine MRI referrals from general practice.<\/jats:p>\n <\/jats:sec>\n \n Methods<\/jats:title>\n Three LLMs (GPT-4, Claude Opus, Gemini) were tasked with assigning an outcome (Accept \u2013 Routine, Accept \u2013 Urgent, Reject) and flagging MRI contraindications for lumbar spine referrals. Three prompts of increasing detail, including clinical guidelines and training examples, were used. Two radiology registrars synthesised 120 referrals, vetted by two board-certified radiologists, with a third resolving disagreements. Performance was assessed using accuracy, precision, recall and F1 scores.<\/jats:p>\n <\/jats:sec>\n \n Results<\/jats:title>\n \n Inter-rater agreement between radiologists was substantial for vetting outcome (Cohen's\n \u03ba<\/jats:italic>\n \u2009=\u20090.76) and contraindication detection (\n \u03ba<\/jats:italic>\n \u2009=\u20090.68). Claude Opus with the full prompt achieved the highest accuracy (0.86) for vetting outcomes. GPT-4 with the instruction-only prompt achieved the highest F1 score (0.88) for contraindication detection. LLMs completed the task substantially faster than radiologists (9.8\u2009\u00b1\u20091.0 vs 135.0\u2009\u00b1\u200945.0\u2005min).\n <\/jats:p>\n <\/jats:sec>\n \n Conclusions<\/jats:title>\n LLMs demonstrate promising performance in vetting radiological referrals for sciatica, particularly with detailed context. All models identified all urgent referrals, suggesting potential for prioritising vetting worklists and improving timeliness of care.<\/jats:p>\n <\/jats:sec>","DOI":"10.1177\/00369330261441582","type":"journal-article","created":{"date-parts":[[2026,4,16]],"date-time":"2026-04-16T13:04:11Z","timestamp":1776344651000},"page":"43-53","update-policy":"https:\/\/doi.org\/10.1177\/sage-journals-update-policy","source":"Crossref","is-referenced-by-count":1,"title":["Evaluation of large language models with clinical guidance for vetting outpatient magnetic resonance imaging lumbar spine referrals"],"prefix":"10.1177","volume":"71","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-3975-0135","authenticated-orcid":false,"given":"William","family":"Clackett","sequence":"first","affiliation":[{"name":"NHS Tayside"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4565-0827","authenticated-orcid":false,"given":"Hatim","family":"Alsusa","sequence":"additional","affiliation":[{"name":"NHS Tayside"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Hannah","family":"Watson","sequence":"additional","affiliation":[{"name":"Canon Medical Research Europe"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Antanas","family":"Kascenas","sequence":"additional","affiliation":[{"name":"Canon Medical Research Europe"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"David","family":"Scott","sequence":"additional","affiliation":[{"name":"NHS Tayside"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Avinash K","family":"Kanodia","sequence":"additional","affiliation":[{"name":"NHS Tayside"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Oliver T","family":"Barry","sequence":"additional","affiliation":[{"name":"NHS Tayside"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Alison Q","family":"O\u2019Neil","sequence":"additional","affiliation":[{"name":"Canon Medical Research Europe"},{"name":"University of Edinburgh, Edinburgh, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"179","published-online":{"date-parts":[[2026,4,16]]},"reference":[{"key":"e_1_3_2_2_2","unstructured":"Royal College of Radiologists. RCR clinical radiology workforce census 2022."},{"key":"e_1_3_2_3_2","doi-asserted-by":"publisher","DOI":"10.1001\/jama.2019.11456"},{"key":"e_1_3_2_4_2","doi-asserted-by":"publisher","DOI":"10.1177\/02841851221139624"},{"key":"e_1_3_2_5_2","unstructured":"Royal College of Radiologists. Vetting (triaging) and cancellation of inappropriate radiology requests 2021."},{"key":"e_1_3_2_6_2","doi-asserted-by":"publisher","DOI":"10.1148\/radiol.230725"},{"key":"e_1_3_2_7_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.crad.2023.11.011"},{"key":"e_1_3_2_8_2","unstructured":"Kaplan J McCandlish S Henighan T et al. Scaling laws for neural language models. arXiv 2020; arXiv:2001.08361."},{"key":"e_1_3_2_9_2","unstructured":"Chiang WL Zheng L Sheng Y et al. Chatbot Arena: an open platform for evaluating LLMs by human preference. In: Proceedings of the Forty-first International Conference on Machine Learning. Vienna. 2024."},{"key":"e_1_3_2_10_2","doi-asserted-by":"publisher","DOI":"10.1186\/s13244-022-01267-8"},{"key":"e_1_3_2_11_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.radi.2022.05.005"},{"key":"e_1_3_2_12_2","doi-asserted-by":"publisher","DOI":"10.1148\/radiol.230970"},{"key":"e_1_3_2_13_2","unstructured":"American College of Radiology. ACR appropriateness criteria. https:\/\/www.acr.org\/Clinical-Resources\/ACR-Appropriateness-Criteria (accessed 22 May 2024)."},{"key":"e_1_3_2_14_2","doi-asserted-by":"publisher","DOI":"10.1053\/j.ro.2023.02.003"},{"key":"e_1_3_2_15_2","unstructured":"Agarwal R Singh A Zhang LM et al. Many-shot in-context learning. arXiv 2024; arXiv:2404.11018."},{"key":"e_1_3_2_16_2","doi-asserted-by":"crossref","unstructured":"Yang H Li M Zhou H et al. One LLM is not enough: harnessing the power of ensemble learning for medical question answering. medRxiv 2023.","DOI":"10.1101\/2023.12.21.23300380"},{"key":"e_1_3_2_17_2","doi-asserted-by":"publisher","DOI":"10.1002\/jmrs.546"},{"key":"e_1_3_2_18_2","doi-asserted-by":"publisher","DOI":"10.1148\/radiol.230806"},{"key":"e_1_3_2_19_2","doi-asserted-by":"publisher","DOI":"10.1148\/radiol.232756"}],"container-title":["Scottish Medical Journal"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/00369330261441582","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/full-xml\/10.1177\/00369330261441582","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/00369330261441582","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,5,26]],"date-time":"2026-05-26T17:47:19Z","timestamp":1779817639000},"score":1,"resource":{"primary":{"URL":"https:\/\/journals.sagepub.com\/doi\/10.1177\/00369330261441582"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,4,16]]},"references-count":18,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2026,5]]}},"alternative-id":["10.1177\/00369330261441582"],"URL":"https:\/\/doi.org\/10.1177\/00369330261441582","relation":{},"ISSN":["0036-9330","2045-6441"],"issn-type":[{"value":"0036-9330","type":"print"},{"value":"2045-6441","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,4,16]]}}}