{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,24]],"date-time":"2026-10-24T02:11:31Z","timestamp":1792807891129,"version":"4.5.0"},"reference-count":20,"publisher":"Springer Science and Business Media LLC","issue":"3","license":[{"start":{"date-parts":[[2025,1,10]],"date-time":"2025-01-10T00:00:00Z","timestamp":1736467200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2025,1,10]],"date-time":"2025-01-10T00:00:00Z","timestamp":1736467200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100014584","name":"Universit\u00e4tsmedizin der Johannes Gutenberg-Universit\u00e4t Mainz","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100014584","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Eur Arch Otorhinolaryngol"],"published-print":{"date-parts":[[2025,3]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:sec>\n                    <jats:title>Introduction<\/jats:title>\n                    <jats:p>Tumor boards are a cornerstone of modern cancer treatment. Given their advanced capabilities, the role of Large Language Models (LLMs) in generating tumor board decisions for otorhinolaryngology (ORL) head and neck surgery is gaining increasing attention. However, concerns over data protection and the use of confidential patient information in web-based LLMs have restricted their widespread adoption and hindered the exploration of their full potential. In this first study of its kind we compared standard human multidisciplinary tumor board recommendations (MDT) against a web-based LLM (ChatGPT-4o) and a locally run LLM (Llama 3) addressing data protection concerns.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Material and methods<\/jats:title>\n                    <jats:p>Twenty-five simulated tumor board cases were presented to an MDT composed of specialists from otorhinolaryngology, craniomaxillofacial surgery, medical oncology, radiology, radiation oncology, and pathology. This multidisciplinary team provided a comprehensive analysis of the cases. The same cases were input into ChatGPT-4o and Llama\u00a03 using structured prompts, and the concordance between the LLMs' and MDT\u2019s recommendations was assessed. Four MDT members evaluated the LLMs' recommendations in terms of medical adequacy (using a six-point Likert scale) and whether the information provided could have influenced the MDT's original recommendations.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>ChatGPT-4o showed 84% concordance (21 out of 25 cases) and Llama 3 demonstrated 92% concordance (23 out of 25 cases) with the MDT in distinguishing between curative and palliative treatment strategies. In 64% of cases (16\/25) ChatGPT-4o and in 60% of cases (15\/25) Llama, identified all first-line therapy options considered by the MDT, though with varying priority. ChatGPT-4o presented all the MDT\u2019s first-line therapies in 52% of cases (13\/25), while Llama 3 offered a homologous treatment strategy in 48% of cases (12\/25). Additionally, both models proposed at least one of the MDT's first-line therapies as their top recommendation in 28% of cases (7\/25). The ratings for medical adequacy yielded a mean score of 4.7 (IQR: 4\u20136) for ChatGPT-4o and 4.3 (IQR: 3\u20135) for Llama 3. In 17% of the assessments (33\/200), MDT members indicated that the LLM recommendations could potentially enhance the MDT's decisions.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Discussion<\/jats:title>\n                    <jats:p>This study demonstrates the capability of both LLMs to provide viable therapeutic recommendations in ORL head and neck surgery. Llama 3, operating locally, bypasses many data protection issues and shows promise as a clinical tool to support MDT decisions. However at present, LLMs should augment rather than replace human decision-making.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.1007\/s00405-024-09153-3","type":"journal-article","created":{"date-parts":[[2025,1,10]],"date-time":"2025-01-10T04:29:07Z","timestamp":1736483347000},"page":"1593-1607","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":24,"title":["Assessment of decision-making with locally run and web-based large language models versus human board recommendations in otorhinolaryngology, head and neck surgery"],"prefix":"10.1007","volume":"282","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-9551-2310","authenticated-orcid":false,"given":"Christoph Raphael","family":"Buhr","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Benjamin Philipp","family":"Ernst","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Andrew","family":"Blaikie","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Harry","family":"Smith","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Tom","family":"Kelsey","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Christoph","family":"Matthias","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Maximilian","family":"Fleischmann","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Florian","family":"Jungmann","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"J\u00fcrgen","family":"Alt","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Christian","family":"Brandts","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Peer W.","family":"K\u00e4mmerer","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sebastian","family":"Foersch","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sebastian","family":"Kuhn","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jonas","family":"Eckrich","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2025,1,10]]},"reference":[{"issue":"3","key":"9153_CR1","doi-asserted-by":"publisher","first-page":"375","DOI":"10.2217\/fon-2021-0471","volume":"18","author":"MS Mano","year":"2021","unstructured":"Mano MS, \u00c7itaku FT, Barach P (2021) Implementing multidisciplinary tumor boards in oncology: a narrative review. Future Oncol 18(3):375\u2013384","journal-title":"Future Oncol"},{"key":"9153_CR2","doi-asserted-by":"publisher","first-page":"9363","DOI":"10.2147\/CMAR.S220976","volume":"12","author":"R Berardi","year":"2020","unstructured":"Berardi R, Morgese F, Rinaldi S, Torniai M, Mentrasti G, Scortichini L et al (2020) <p>benefits and limitations of a multidisciplinary approach in cancer patient management<\/p>. Cancer Manag Res 12:9363\u20139374","journal-title":"Cancer Manag Res"},{"issue":"11","key":"9153_CR3","doi-asserted-by":"publisher","first-page":"3041","DOI":"10.3390\/jcm13113041","volume":"13","author":"SM Pressman","year":"2024","unstructured":"Pressman SM, Borna S, Gomez-Cabello CA, Haider SA, Haider CR, Forte AJ (2024) Clinical and surgical applications of large language models: a systematic review. J Clin Med 13(11):3041","journal-title":"J Clin Med"},{"key":"9153_CR4","doi-asserted-by":"publisher","first-page":"e48568","DOI":"10.2196\/48568","volume":"25","author":"J Liu","year":"2023","unstructured":"Liu J, Wang C, Liu S (2023) Utility of ChatGPT in clinical practice. J Med Internet Res 25:e48568","journal-title":"J Med Internet Res"},{"key":"9153_CR5","doi-asserted-by":"publisher","first-page":"e49183","DOI":"10.2196\/49183","volume":"9","author":"CR Buhr","year":"2023","unstructured":"Buhr CR, Smith H, Huppertz T, Bahr-Hamm K, Matthias C, Blaikie A et al (2023) ChatGPT versus consultants: blinded evaluation on answering otorhinolaryngology case-based questions. JMIR Med Educ 9:e49183","journal-title":"JMIR Med Educ"},{"issue":"3","key":"9153_CR6","doi-asserted-by":"publisher","first-page":"237","DOI":"10.1080\/00016489.2024.2352843","volume":"144","author":"CR Buhr","year":"2024","unstructured":"Buhr CR, Smith H, Huppertz T, Bahr-Hamm K, Matthias C, Cuny C et al (2024) Assessing unknown potential\u2014quality and limitations of different large language models in the field of otorhinolaryngology. Acta Otolaryngol 144(3):237\u2013242","journal-title":"Acta Otolaryngol"},{"key":"9153_CR7","doi-asserted-by":"publisher","unstructured":"Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN et al (2017) Attention is all you need. Advances in neural information processing systems. https:\/\/doi.org\/10.48550\/arXiv.1706.03762","DOI":"10.48550\/arXiv.1706.03762"},{"key":"9153_CR8","unstructured":"Radford A, Narasimhan K, Salimans T, Sutskever I (2018) Improving language understanding by generative pre-training. https:\/\/cdn.openai.com\/research-covers\/language-unsupervised\/language_understanding_paper.pdf"},{"issue":"1","key":"9153_CR9","doi-asserted-by":"publisher","first-page":"120","DOI":"10.1038\/s41746-023-00873-0","volume":"6","author":"B Mesk\u00f3","year":"2023","unstructured":"Mesk\u00f3 B, Topol EJ (2023) The imperative for regulatory oversight of large language models (or generative AI) in healthcare. NPJ Digit Med 6(1):120. https:\/\/doi.org\/10.1038\/s41746-023-00873-0","journal-title":"NPJ Digit Med"},{"key":"9153_CR10","doi-asserted-by":"publisher","first-page":"53","DOI":"10.1016\/j.coisb.2017.07.001","volume":"4","author":"BM Knoppers","year":"2017","unstructured":"Knoppers BM, Thorogood AM (2017) Ethics and big data in health. Curr Opin Syst Biol 4:53\u201357","journal-title":"Curr Opin Syst Biol"},{"issue":"23","key":"9153_CR11","doi-asserted-by":"publisher","first-page":"2202","DOI":"10.1056\/NEJMsa1713258","volume":"378","author":"MM Mello","year":"2018","unstructured":"Mello MM, Lieou V, Goodman SN (2018) Clinical trial participants\u2019 views of the risks and benefits of data sharing. N Engl J Med 378(23):2202\u20132211","journal-title":"N Engl J Med"},{"issue":"1","key":"9153_CR12","doi-asserted-by":"publisher","first-page":"37","DOI":"10.1038\/s41591-018-0272-7","volume":"25","author":"WN Price","year":"2019","unstructured":"Price WN, Cohen IG (2019) Privacy in the age of medical big data. Nat Med 25(1):37\u201343","journal-title":"Nat Med"},{"key":"9153_CR13","doi-asserted-by":"publisher","DOI":"10.1007\/s00117-024-01327-8","volume-title":"Gro\u00dfe Sprachmodelle von OpenAI, Google, Meta, X und Co","author":"S Nowak","year":"2024","unstructured":"Nowak S, Sprinkart AM (2024) Gro\u00dfe Sprachmodelle von OpenAI, Google, Meta, X und Co. Die Radiologie"},{"issue":"1","key":"9153_CR14","doi-asserted-by":"publisher","first-page":"44","DOI":"10.1038\/s41523-023-00557-8","volume":"9","author":"V Sorin","year":"2023","unstructured":"Sorin V, Klang E, Sklair-Levy M, Cohen I, Zippel DB, Balint Lahat N et al (2023) Large language model (ChatGPT) as a support tool for breast tumor board. NPJ Breast Cancer 9(1):44. https:\/\/doi.org\/10.1038\/s41523-023-00557-8","journal-title":"NPJ Breast Cancer"},{"issue":"10","key":"9153_CR15","doi-asserted-by":"publisher","first-page":"1502","DOI":"10.3390\/jpm13101502","volume":"13","author":"S Griewing","year":"2023","unstructured":"Griewing S, Gremke N, Wagner U, Lingenfelder M, Kuhn S, Boekhoff J (2023) Challenging ChatGPT 3.5 in senology\u2014an assessment of concordance with breast cancer tumor board decision making. J Pers Med 13(10):1502. https:\/\/doi.org\/10.3390\/jpm13101502","journal-title":"J Pers Med"},{"issue":"4","key":"9153_CR16","doi-asserted-by":"publisher","first-page":"2105","DOI":"10.1007\/s00405-023-08326-w","volume":"281","author":"JR Lechien","year":"2023","unstructured":"Lechien JR, Chiesa-Estomba C-M, Baudouin R, Hans S (2023) Accuracy of ChatGPT in head and neck oncological board decisions: preliminary findings. Eur Arch Otorhinolaryngol 281(4):2105\u20132114","journal-title":"Eur Arch Otorhinolaryngol"},{"key":"9153_CR17","doi-asserted-by":"publisher","first-page":"1353031","DOI":"10.3389\/fonc.2024.1353031","volume":"14","author":"B Schmidl","year":"2024","unstructured":"Schmidl B, H\u00fctten T, Pigorsch S, St\u00f6gbauer F, Hoch CC, Hussain T et al (2024) Assessing the role of advanced artificial intelligence as a tool in multidisciplinary tumor board decision-making for primary head and neck cancer cases. Front Oncol 14:1353031. https:\/\/doi.org\/10.3389\/fonc.2024.1353031","journal-title":"Front Oncol"},{"issue":"11","key":"9153_CR18","doi-asserted-by":"publisher","first-page":"6099","DOI":"10.1007\/s00405-024-08828-1","volume":"281","author":"B Schmidl","year":"2024","unstructured":"Schmidl B, H\u00fctten T, Pigorsch S, St\u00f6gbauer F, Hoch CC, Hussain T et al (2024) Assessing the use of the novel tool Claude 3 in comparison to ChatGPT 4.0 as an artificial intelligence tool in the diagnosis and therapy of primary head and neck cancer cases. Eur Arch Otorhinolaryngol 281(11):6099\u20136109. https:\/\/doi.org\/10.1007\/s00405-024-08828-1","journal-title":"Eur Arch Otorhinolaryngol"},{"issue":"11","key":"9153_CR19","doi-asserted-by":"publisher","first-page":"3098","DOI":"10.1038\/s41591-024-03180-7","volume":"30","author":"M Reis","year":"2024","unstructured":"Reis M, Reis F, Kunde W (2024) Influence of believed AI involvement on the perception of digital medical advice. Nat\nMed 30(11):3098\u20133100. https:\/\/doi.org\/10.1038\/s41591-024-03180-7","journal-title":"Nat Med"},{"issue":"1","key":"9153_CR20","doi-asserted-by":"publisher","first-page":"100485","DOI":"10.1016\/j.xcrm.2021.100485","volume":"3","author":"G Bazoukis","year":"2022","unstructured":"Bazoukis G, Hall J, Loscalzo J, Antman EM, Fuster V, Armoundas AA (2022) The inclusion of augmented intelligence in medicine: a framework for successful implementation. Cell Rep Med 3(1):100485. https:\/\/doi.org\/10.1016\/j.xcrm.2021.100485","journal-title":"Cell Rep Med"}],"container-title":["European Archives of Oto-Rhino-Laryngology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s00405-024-09153-3.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s00405-024-09153-3\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s00405-024-09153-3.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,3,8]],"date-time":"2025-03-08T03:32:13Z","timestamp":1741404733000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s00405-024-09153-3"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,1,10]]},"references-count":20,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2025,3]]}},"alternative-id":["9153"],"URL":"https:\/\/doi.org\/10.1007\/s00405-024-09153-3","relation":{},"ISSN":["0937-4477","1434-4726"],"issn-type":[{"value":"0937-4477","type":"print"},{"value":"1434-4726","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,1,10]]},"assertion":[{"value":"26 September 2024","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"9 December 2024","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"10 January 2025","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}}]}}