{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,29]],"date-time":"2026-08-29T19:16:02Z","timestamp":1788030962449,"version":"build-2784847793"},"reference-count":23,"publisher":"Wiley","issue":"3","license":[{"start":{"date-parts":[[2026,5,29]],"date-time":"2026-05-29T00:00:00Z","timestamp":1780012800000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/onlinelibrary.wiley.com\/termsAndConditions#vor"},{"start":{"date-parts":[[2026,5,29]],"date-time":"2026-05-29T00:00:00Z","timestamp":1780012800000},"content-version":"tdm","delay-in-days":0,"URL":"http:\/\/doi.wiley.com\/10.1002\/tdm_license_1.1"}],"content-domain":{"domain":["aao-hnsfjournals.onlinelibrary.wiley.com"],"crossmark-restriction":true},"short-container-title":["Otolaryngol.--head neck surg."],"published-print":{"date-parts":[[2026,9]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:sec>\n                    <jats:title>Objectives<\/jats:title>\n                    <jats:p>Artificial Intelligence (AI) is increasingly integrated into medicine, including otolaryngology. However, concerns remain regarding the accuracy of generated content and the tendency of large language models (LLMs) to fabricate references. This study evaluates the accuracy, appropriateness, readability, and hallucination of references in 2 prevalent large language models, ChatGPT and Claude, in response to common otolaryngological questions.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Study Design<\/jats:title>\n                    <jats:p>Prospective observational study.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Settings<\/jats:title>\n                    <jats:p>Academic tertiary care center.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Methods<\/jats:title>\n                    <jats:p>Thirty\u2010six otolaryngologic questions were individually entered into ChatGPT 4.0 Plus and CLAUDE in separate sessions, with explicit instructions to avoid utilizing previous memories. To assess reproducibility, each query was submitted twice. Two otolaryngologists independently rated the accuracy of responses. Readability was evaluated using the Flesch Reading Ease (FRE) score. Reference hallucinations were assessed by analyzing the reference validity and relevance.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>ChatGPT and CLAUDE had an FRE of 47 and 25.2 out of 100, respectively. For patient readability, ChatGPT scored a 3.60 while Claude scored a 4.68 out of 5. Claude scored slightly higher on accuracy, receiving a score of 4.42 out of 5 while ChatGPT received a 3.81. Both models hallucinated at least half of their references, with some citations irrelevant or incorrectly formatted. Thematic analysis revealed frequent vagueness, poor clinical prioritization, and excessive jargon across both models.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusion<\/jats:title>\n                    <jats:p>Both ChatGPT and CLAUDE often produced partially inaccurate, jargon\u2010filled responses and failed to consistently provide valid references when answering common otolaryngologic patient questions. Our results highlight the need for better understanding and regulation of LLM limitations in clinical and patient\u2010facing applications.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.1002\/ohn.70309","type":"journal-article","created":{"date-parts":[[2026,5,29]],"date-time":"2026-05-29T10:55:51Z","timestamp":1780052151000},"page":"594-602","update-policy":"https:\/\/doi.org\/10.1002\/crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Large Language Model Responses to Common Otolaryngological Questions: Evaluating Accuracy, Appropriateness, Readability, and Hallucinations"],"prefix":"10.1002","volume":"175","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-7612-0285","authenticated-orcid":false,"given":"Mitali","family":"Sakharkar","sequence":"first","affiliation":[{"name":"Boston University Chobanian and Avedisian School of Medicine Boston Massachusetts USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Parth","family":"Jalihal","sequence":"additional","affiliation":[{"name":"Boston University Chobanian and Avedisian School of Medicine Boston Massachusetts USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4846-2234","authenticated-orcid":false,"given":"Sarah","family":"Chang","sequence":"additional","affiliation":[{"name":"Boston University Chobanian and Avedisian School of Medicine Boston Massachusetts USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Prachi","family":"Patel","sequence":"additional","affiliation":[{"name":"Boston University Department of Otolaryngology Boston Massachusetts USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Melissa","family":"Chavez","sequence":"additional","affiliation":[{"name":"Boston University Department of Otolaryngology Boston Massachusetts USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7329-8560","authenticated-orcid":false,"given":"Jessica","family":"Levi","sequence":"additional","affiliation":[{"name":"Boston University Chobanian and Avedisian School of Medicine Boston Massachusetts USA"},{"name":"Boston University Department of Otolaryngology Boston Massachusetts USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"311","published-online":{"date-parts":[[2026,5,29]]},"reference":[{"key":"e_1_2_9_2_1","doi-asserted-by":"publisher","DOI":"10.3389\/frai.2023.1169595"},{"key":"e_1_2_9_3_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.cmpbup.2024.100145"},{"key":"e_1_2_9_4_1","doi-asserted-by":"publisher","DOI":"10.1016\/S2589-7500(23)00083-3"},{"key":"e_1_2_9_5_1","first-page":"1","article-title":"Ethical issues of artificial intelligence in medicine and healthcare","volume":"50","author":"Farhud DD","year":"2021","journal-title":"Iran J Publ Health"},{"key":"e_1_2_9_6_1","doi-asserted-by":"publisher","DOI":"10.1038\/s41591-023-02448-8"},{"key":"e_1_2_9_7_1","doi-asserted-by":"crossref","unstructured":"HuangL YuW MaW et al.A survey on hallucination in large language models: principles taxonomy challenges and open questions.ACM TransInf Syst.2025;43(2): Article 42.doi:10.1145\/3703155","DOI":"10.1145\/3703155"},{"key":"e_1_2_9_8_1","volume-title":"What Is ChatGPT Doing:\u2026 And Why Does It Work?","author":"Wolfram S","year":"2023"},{"key":"e_1_2_9_9_1","doi-asserted-by":"publisher","DOI":"10.1093\/pnasnexus\/pgae346"},{"key":"e_1_2_9_10_1","doi-asserted-by":"publisher","DOI":"10.1186\/s13000-024-01464-7"},{"key":"e_1_2_9_11_1","doi-asserted-by":"publisher","DOI":"10.3389\/fmed.2025.1527864"},{"key":"e_1_2_9_12_1","doi-asserted-by":"publisher","DOI":"10.1007\/s00405-023-08441-8"},{"key":"e_1_2_9_13_1","doi-asserted-by":"publisher","DOI":"10.1002\/ohn.807"},{"key":"e_1_2_9_14_1","doi-asserted-by":"publisher","DOI":"10.1007\/s00405-023-08423-w"},{"key":"e_1_2_9_15_1","doi-asserted-by":"publisher","DOI":"10.1007\/s00405-023-08205-4"},{"key":"e_1_2_9_16_1","doi-asserted-by":"publisher","DOI":"10.1080\/00016489.2023.2254809"},{"key":"e_1_2_9_17_1","doi-asserted-by":"publisher","DOI":"10.1002\/oto2.94"},{"key":"e_1_2_9_18_1","doi-asserted-by":"publisher","DOI":"10.1002\/lary.31410"},{"key":"e_1_2_9_19_1","doi-asserted-by":"publisher","DOI":"10.1007\/s00464-024-10739-5"},{"key":"e_1_2_9_20_1","unstructured":"WeisB.Health Literacy: A Manual for Clinicians. American Medical Association Foundation and American Medical Association. 2003;2024."},{"key":"e_1_2_9_21_1","volume-title":"Plain Language at NIH","author":"National Institutes of Health","year":"2016"},{"key":"e_1_2_9_22_1","doi-asserted-by":"crossref","unstructured":"RenzeM GuvenE. The effect of sampling temperature on problem solving in large language models. In:Findings of the Association for Computational Linguistics: EMNLP 2024.2024:7346\u20107356.doi:10.18653\/v1\/2024.findings-emnlp.432","DOI":"10.18653\/v1\/2024.findings-emnlp.432"},{"key":"e_1_2_9_23_1","doi-asserted-by":"publisher","DOI":"10.2196\/56764"},{"key":"e_1_2_9_24_1","volume-title":"Confronting Health Misinformation: The U.S. Surgeon General's Advisory on Building a Healthy Information Environment [Internet]","author":"Office of the Surgeon General (OSG)","year":"2021"}],"container-title":["Otolaryngology\u2013Head and Neck Surgery"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/aao-hnsfjournals.onlinelibrary.wiley.com\/doi\/pdf\/10.1002\/ohn.70309","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/aao-hnsfjournals.onlinelibrary.wiley.com\/doi\/full-xml\/10.1002\/ohn.70309","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/aao-hnsfjournals.onlinelibrary.wiley.com\/doi\/pdf\/10.1002\/ohn.70309","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,8,29]],"date-time":"2026-08-29T18:37:32Z","timestamp":1788028652000},"score":1,"resource":{"primary":{"URL":"https:\/\/aao-hnsfjournals.onlinelibrary.wiley.com\/doi\/10.1002\/ohn.70309"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,5,29]]},"references-count":23,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2026,9]]}},"alternative-id":["10.1002\/ohn.70309"],"URL":"https:\/\/doi.org\/10.1002\/ohn.70309","archive":["Portico"],"relation":{},"ISSN":["0194-5998","1097-6817"],"issn-type":[{"value":"0194-5998","type":"print"},{"value":"1097-6817","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,5,29]]},"assertion":[{"value":"2025-07-08","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2026-04-19","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2026-05-29","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}