{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,14]],"date-time":"2026-10-14T16:12:41Z","timestamp":1791994361167,"version":"4.4.0"},"reference-count":12,"publisher":"Springer Science and Business Media LLC","issue":"9","license":[{"start":{"date-parts":[[2024,5,25]],"date-time":"2024-05-25T00:00:00Z","timestamp":1716595200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2024,5,25]],"date-time":"2024-05-25T00:00:00Z","timestamp":1716595200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/100012352","name":"Universit\u00e0 degli Studi di Milano","doi-asserted-by":"crossref","id":[{"id":"10.13039\/100012352","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Eur Arch Otorhinolaryngol"],"published-print":{"date-parts":[[2024,9]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:sec>\n                <jats:title>Purpose<\/jats:title>\n                <jats:p>This study evaluates the efficacy of two advanced Large Language Models (LLMs), OpenAI\u2019s ChatGPT 4 and Google\u2019s Gemini Advanced, in providing treatment recommendations for head and neck oncology cases. The aim is to assess their utility in supporting multidisciplinary oncological evaluations and decision-making processes.<\/jats:p>\n              <\/jats:sec><jats:sec>\n                <jats:title>Methods<\/jats:title>\n                <jats:p>This comparative analysis examined the\u00a0responses of ChatGPT 4 and Gemini Advanced to five hypothetical cases of head and neck cancer, each representing a different anatomical subsite. The responses were evaluated against the latest National Comprehensive Cancer Network (NCCN) guidelines by two blinded panels using the total disagreement score (TDS) and the artificial intelligence performance instrument (AIPI). Statistical assessments were performed using the Wilcoxon signed-rank test and the Friedman test.<\/jats:p>\n              <\/jats:sec><jats:sec>\n                <jats:title>Results<\/jats:title>\n                <jats:p>Both LLMs produced relevant treatment recommendations with ChatGPT 4 generally outperforming Gemini Advanced regarding adherence to guidelines and comprehensive treatment planning. ChatGPT 4 showed higher AIPI scores (median 3 [2\u20134]) compared to Gemini Advanced (median 2 [2\u20133]), indicating better overall performance. Notably, inconsistencies were observed in the management of induction chemotherapy and surgical decisions, such as neck dissection.<\/jats:p>\n              <\/jats:sec><jats:sec>\n                <jats:title>Conclusions<\/jats:title>\n                <jats:p>While both LLMs demonstrated the potential to aid in the multidisciplinary management of head and neck oncology, discrepancies in certain critical areas highlight the need for further refinement. The study supports the growing role of AI in enhancing clinical decision-making but also emphasizes the necessity for continuous updates and validation against current clinical standards to integrate AI into healthcare practices fully.<\/jats:p>\n              <\/jats:sec>","DOI":"10.1007\/s00405-024-08746-2","type":"journal-article","created":{"date-parts":[[2024,5,25]],"date-time":"2024-05-25T13:01:57Z","timestamp":1716642117000},"page":"5001-5006","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":40,"title":["Reliability of large language models for advanced head and neck malignancies management: a comparison between ChatGPT 4 and Gemini Advanced"],"prefix":"10.1007","volume":"281","author":[{"given":"Andrea","family":"Lorenzi","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0000-6979-4206","authenticated-orcid":false,"given":"Giorgia","family":"Pugliese","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Antonino","family":"Maniaci","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jerome R.","family":"Lechien","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Fabiana","family":"Allevi","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Paolo","family":"Boscolo-Rizzo","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Luigi Angelo","family":"Vaira","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Alberto Maria","family":"Saibene","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2024,5,25]]},"reference":[{"key":"8746_CR1","doi-asserted-by":"publisher","first-page":"1237","DOI":"10.1093\/jamia\/ocad072","volume":"30","author":"S Liu","year":"2023","unstructured":"Liu S et al (2023) Using AI-generated suggestions from ChatGPT to optimize clinical decision support. J Am Med Inform Assoc 30:1237\u20131245","journal-title":"J Am Med Inform Assoc"},{"key":"8746_CR2","doi-asserted-by":"publisher","first-page":"2123","DOI":"10.1007\/s00405-024-08525-z","volume":"281","author":"F Marchi","year":"2024","unstructured":"Marchi F, Bellini E, Iandelli A, Sampieri C, Peretti G (2024) Exploring the landscape of AI-assisted decision-making in head and neck cancer treatment: a comparative analysis of NCCN guidelines and ChatGPT responses. Eur Arch Otorhinolaryngol 281:2123\u20132136","journal-title":"Eur Arch Otorhinolaryngol"},{"key":"8746_CR3","doi-asserted-by":"publisher","first-page":"1425","DOI":"10.1007\/s12070-023-04201-6","volume":"76","author":"G Sarma","year":"2024","unstructured":"Sarma G, Kashyap H, Medhi PP (2024) ChatGPT in head and neck oncology-opportunities and challenges. Indian J Otolaryngol Head Neck Surg 76:1425\u20131429","journal-title":"Indian J Otolaryngol Head Neck Surg"},{"key":"8746_CR4","doi-asserted-by":"publisher","first-page":"1835","DOI":"10.1007\/s00405-023-08372-4","volume":"281","author":"AM Saibene","year":"2024","unstructured":"Saibene AM et al (2024) Reliability of large language models in managing odontogenic sinusitis clinical scenarios: a preliminary multidisciplinary evaluation. Eur Arch Otorhinolaryngol 281:1835\u20131841","journal-title":"Eur Arch Otorhinolaryngol"},{"key":"8746_CR5","doi-asserted-by":"publisher","DOI":"10.1002\/ohn.489","author":"LA Vaira","year":"2023","unstructured":"Vaira LA et al (2023) Accuracy of ChatGPT-generated information on head and neck and oromaxillofacial surgery: a multicenter collaborative analysis. Otolaryngol Head Neck Surg. https:\/\/doi.org\/10.1002\/ohn.489","journal-title":"Otolaryngol Head Neck Surg"},{"key":"8746_CR6","doi-asserted-by":"publisher","DOI":"10.1002\/ohn.759","author":"JR Lechien","year":"2024","unstructured":"Lechien JR et al (2024) Performance and consistency of ChatGPT-4 versus otolaryngologists: a clinical case series. Otolaryngol Head Neck Surg. https:\/\/doi.org\/10.1002\/ohn.759","journal-title":"Otolaryngol Head Neck Surg"},{"key":"8746_CR7","doi-asserted-by":"publisher","unstructured":"Vaira LA et al. QAMAI. Eur. Arch. Otorhinolaryngol. (being published) https:\/\/doi.org\/10.1007\/s00405-024-08710-0","DOI":"10.1007\/s00405-024-08710-0"},{"key":"8746_CR8","doi-asserted-by":"publisher","first-page":"125","DOI":"10.1007\/s10439-023-03288-w","volume":"52","author":"Z Liao","year":"2024","unstructured":"Liao Z, Wang J, Shi Z, Lu L, Tabata H (2024) Revolutionary potential of ChatGPT in constructing intelligent clinical decision support systems. Ann Biomed Eng 52:125\u2013129","journal-title":"Ann Biomed Eng"},{"key":"8746_CR9","doi-asserted-by":"publisher","first-page":"3360","DOI":"10.1007\/s12325-023-02527-9","volume":"40","author":"AA M\u00e4kitie","year":"2023","unstructured":"M\u00e4kitie AA et al (2023) Artificial intelligence in head and neck cancer: a systematic review of systematic reviews. Adv Ther 40:3360\u20133380","journal-title":"Adv Ther"},{"key":"8746_CR10","doi-asserted-by":"publisher","first-page":"529","DOI":"10.1007\/s00405-022-07701-3","volume":"280","author":"AM Bulfamante","year":"2023","unstructured":"Bulfamante AM et al (2023) Artificial intelligence, machine learning, and deep learning in rhinology: a systematic review. Eur Arch Otorhinolaryngol 280:529\u2013542","journal-title":"Eur Arch Otorhinolaryngol"},{"key":"8746_CR11","doi-asserted-by":"publisher","first-page":"2051","DOI":"10.1007\/s00405-024-08579-z","volume":"281","author":"JR Lechien","year":"2024","unstructured":"Lechien JR (2024) Generative artificial intelligence in otolaryngology-head and neck surgery editorial: be an actor of the future or follower. Eur Arch Otorhinolaryngol 281:2051\u20132053","journal-title":"Eur Arch Otorhinolaryngol"},{"key":"8746_CR12","doi-asserted-by":"publisher","first-page":"2063","DOI":"10.1007\/s00405-023-08219-y","volume":"281","author":"JR Lechien","year":"2024","unstructured":"Lechien JR et al (2024) Validity and reliability of an instrument evaluating the performance of intelligent chatbot: the Artificial Intelligence Performance Instrument (AIPI). Eur Arch Otorhinolaryngol 281:2063\u20132079","journal-title":"Eur Arch Otorhinolaryngol"}],"container-title":["European Archives of Oto-Rhino-Laryngology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s00405-024-08746-2.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s00405-024-08746-2\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s00405-024-08746-2.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,9,12]],"date-time":"2024-09-12T21:10:38Z","timestamp":1726175438000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s00405-024-08746-2"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,5,25]]},"references-count":12,"journal-issue":{"issue":"9","published-print":{"date-parts":[[2024,9]]}},"alternative-id":["8746"],"URL":"https:\/\/doi.org\/10.1007\/s00405-024-08746-2","relation":{},"ISSN":["0937-4477","1434-4726"],"issn-type":[{"value":"0937-4477","type":"print"},{"value":"1434-4726","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,5,25]]},"assertion":[{"value":"16 May 2024","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"17 May 2024","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"25 May 2024","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors have no potential conflict of interest or financial disclosures pertaining to this article.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}},{"value":"This study did not involve human participants, their data, or biological material. Therefore, it did not require institutional research ethics committee evaluation.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethics approval"}}]}}