{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,12]],"date-time":"2026-08-12T07:37:32Z","timestamp":1786520252098,"version":"build-2736575974"},"reference-count":24,"publisher":"Baishideng Publishing Group Inc.","issue":"3","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["World J Gastroenterol"],"abstract":"<jats:p>BACKGROUND<\/jats:p>\n          <jats:p>Patients with hepatitis B virus (HBV) infection require chronic and personalized care to improve outcomes. Large language models (LLMs) can potentially provide medical information for patients.<\/jats:p>\n          <jats:p>AIM<\/jats:p>\n          <jats:p>To examine the performance of three LLMs, ChatGPT-3.5, ChatGPT-4.0, and Google Gemini, in answering HBV-related questions.<\/jats:p>\n          <jats:p>METHODS<\/jats:p>\n          <jats:p>LLMs\u2019 responses to HBV-related questions were independently graded by two medical professionals using a four-point accuracy scale, and disagreements were resolved by a third reviewer. Each question was run three times using three LLMs. Readability was assessed via  the Gunning Fog index and Flesch-Kincaid grade level.<\/jats:p>\n          <jats:p>RESULTS<\/jats:p>\n          <jats:p>Overall, all three LLM chatbots achieved high average accuracy scores for subjective questions (ChatGPT-3.5: 3.50; ChatGPT-4.0: 3.69; Google Gemini: 3.53, out of a maximum score of 4). With respect to objective questions, ChatGPT-4.0 achieved an 80.8% accuracy rate, compared with 62.9% for ChatGPT-3.5 and 73.1% for Google Gemini. Across the six domains, ChatGPT-4.0 performed better in terms of diagnosis, whereas Google Gemini demonstrated excellent clinical manifestations. Notably, in the readability analysis, the mean Gunning Fog index and Flesch-Kincaid grade level scores of the three LLM chatbots were significantly higher than the standard level eight, far exceeding the reading level of the normal population.<\/jats:p>\n          <jats:p>CONCLUSION<\/jats:p>\n          <jats:p>Our results highlight the potential of LLMs, especially ChatGPT-4.0, for delivering responses to HBV-related questions. LLMs may be an adjunctive informational tool for patients and physicians to improve outcomes. Nevertheless, current LLMs should not replace personalized treatment recommendations from physicians in the management of HBV infection.<\/jats:p>","DOI":"10.3748\/wjg.v31.i3.101092","type":"journal-article","created":{"date-parts":[[2024,12,17]],"date-time":"2024-12-17T08:18:05Z","timestamp":1734423485000},"source":"Crossref","is-referenced-by-count":9,"title":["Exploring the performance of large language models on hepatitis B infection-related questions: A comparative study"],"prefix":"10.3748","volume":"31","author":[{"given":"Yu","family":"Li","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Chen-Kai","family":"Huang","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yi","family":"Hu","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Xiao-Dong","family":"Zhou","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Cong","family":"He","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jia-Wei","family":"Zhong","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"2060","published-online":{"date-parts":[[2025,1,21]]},"reference":[{"key":"B1","unstructured":"World Health Organization.   Global progress report on HIV, viral hepatitis and sexually transmitted infections, 2021. [cited 30 July 2024]. Available from: https:\/\/www.who.int\/publications\/i\/item\/9789240027077"},{"key":"B2","unstructured":"World Health Organization.   Global Hepatitis Report, 2017. [cited 30 July 2024]. Available from: https:\/\/www.who.int\/publications\/i\/item\/9789241565455"},{"key":"B3","doi-asserted-by":"publisher","DOI":"10.1038\/s41575-023-00760-9"},{"key":"B4","doi-asserted-by":"publisher","DOI":"10.1056\/NEJMsr2214184"},{"key":"B5","doi-asserted-by":"publisher","DOI":"10.3389\/fpubh.2023.1166120"},{"key":"B6","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pdig.0000198"},{"key":"B7","doi-asserted-by":"publisher","DOI":"10.1038\/s41591-023-02289-5"},{"key":"B8","unstructured":"OpenAI.   Introducing ChatGPT. [cited 30 July 2024]. Available from: https:\/\/openai.com\/index\/chatgpt\/"},{"key":"B9","doi-asserted-by":"publisher","DOI":"10.3350\/cmh.2023.0089"},{"key":"B10","doi-asserted-by":"publisher","DOI":"10.2215\/CJN.0000000000000330"},{"key":"B11","doi-asserted-by":"publisher","DOI":"10.1001\/jama.2013.629"},{"key":"B12","unstructured":"World Health Organization.   Guidelines for the prevention, diagnosis, care and treatment for people with chronic hepatitis B infection. [cited 30 July 2024]. Available from: https:\/\/www.who.int\/publications\/i\/item\/9789240090903"},{"key":"B13","doi-asserted-by":"publisher","DOI":"10.1016\/j.ebiom.2023.104770"},{"key":"B14","doi-asserted-by":"publisher","DOI":"10.1186\/s12967-024-05302-8"},{"key":"B15","doi-asserted-by":"publisher","DOI":"10.1001\/jamaophthalmol.2023.1144"},{"key":"B16","doi-asserted-by":"publisher","DOI":"10.1227\/neu.0000000000002551"},{"key":"B17","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2303.08774"},{"key":"B18","doi-asserted-by":"publisher","DOI":"10.1038\/s41598-023-46995-z"},{"key":"B19","doi-asserted-by":"publisher","DOI":"10.2196\/48002"},{"key":"B20","doi-asserted-by":"publisher","DOI":"10.1016\/j.cgh.2021.07.036"},{"key":"B21","doi-asserted-by":"publisher","DOI":"10.14218\/JCTH.2023.00320"},{"key":"B22","doi-asserted-by":"publisher","DOI":"10.3390\/diagnostics13111950"},{"key":"B23","doi-asserted-by":"publisher","DOI":"10.1007\/s00417-023-06078-1"},{"key":"B24","doi-asserted-by":"publisher","DOI":"10.1038\/s41598-023-50884-w"}],"container-title":["World Journal of Gastroenterology"],"original-title":[],"link":[{"URL":"https:\/\/www.wjgnet.com\/1007-9327\/full\/v31\/i3\/101092.htm","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.wjgnet.com\/1007-9327\/full\/v31\/i3\/101092.htm","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,12,17]],"date-time":"2024-12-17T08:19:42Z","timestamp":1734423582000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.wjgnet.com\/1007-9327\/full\/v31\/i3\/101092.htm"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,1,21]]},"references-count":24,"journal-issue":{"issue":"3"},"alternative-id":["101092"],"URL":"https:\/\/doi.org\/10.3748\/wjg.v31.i3.101092","relation":{},"ISSN":["1007-9327"],"issn-type":[{"value":"1007-9327","type":"print"}],"subject":[],"published":{"date-parts":[[2025,1,21]]}}}