{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,9]],"date-time":"2026-10-09T04:50:18Z","timestamp":1791521418905,"version":"4.3.4"},"reference-count":39,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2026,10,9]],"date-time":"2026-10-09T00:00:00Z","timestamp":1791504000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Cell. Infect. Microbiol."],"abstract":"<jats:sec>\n                    <jats:title>Background<\/jats:title>\n                    <jats:p>Immune checkpoint inhibitor-related liver injury (ICI-LI) has heterogeneous phenotypes, a complex differential diagnosis, and evolving management. Gut microbiome research provides biological plausibility for immune regulation during ICI therapy, but evidence specific to ICI-LI remains limited and is not yet clinically actionable. We evaluated whether publicly accessible large language model (LLM) chatbots can organize this mixed evidence base and communicate its uncertainty.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Methods<\/jats:title>\n                    <jats:p>We developed a disease-specific benchmark of 25 questions covering immunological mechanisms, microbiome and immune regulation, diagnosis, biomarkers, and management of ICI-LI. The benchmark addressed ICI-LI broadly and included a focused microbiome component. Four publicly available LLM platforms were evaluated using DISCERN, EQIP, the Global Quality Score, the JAMA Benchmark Criteria, and six readability indices. These instruments assessed information quality, structure, transparency, and patient-facing readability rather than factual accuracy or clinical decision-making.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>Performance differed across DISCERN, EQIP, and JAMA scores, whereas the Global Quality Score showed limited discrimination. Question-level variation was most apparent for items requiring evidence appraisal or translation. All platforms produced structured responses, but none met the prespecified patient-facing readability benchmark. Readability was interpreted separately from clinician-facing information synthesis.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusions<\/jats:title>\n                    <jats:p>Disease-specific multidimensional evaluation is needed before LLM-generated information can be considered reliable for microbiome-informed clinical communication or decision support. Evaluation should emphasize evidence calibration and distinguish biological plausibility, association, and clinically actionable evidence.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.3389\/fcimb.2026.1955356","type":"journal-article","created":{"date-parts":[[2026,10,9]],"date-time":"2026-10-09T04:25:45Z","timestamp":1791519945000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["Evaluation of large language models for ICI-related liver injury with an emerging microbiome focus"],"prefix":"10.3389","volume":"16","author":[{"given":"Man","family":"Sun","sequence":"first","affiliation":[{"name":"Department of Oncology, The Second Hospital of Dalian Medical University","place":["Dalian, Liaoning, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Dan","family":"Zang","sequence":"additional","affiliation":[{"name":"Department of Oncology, The Second Hospital of Dalian Medical University","place":["Dalian, Liaoning, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Huan","family":"Zhou","sequence":"additional","affiliation":[{"name":"Department of Oncology, The Second Hospital of Dalian Medical University","place":["Dalian, Liaoning, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jun","family":"Chen","sequence":"additional","affiliation":[{"name":"Department of Oncology, The Second Hospital of Dalian Medical University","place":["Dalian, Liaoning, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1965","published-online":{"date-parts":[[2026,10,9]]},"reference":[{"key":"B1","doi-asserted-by":"publisher","first-page":"600","DOI":"10.1038\/s41746-025-01963-x","article-title":"The evaluation illusion of large language models in medicine","volume":"8","author":"Agrawal","year":"2025","journal-title":"NPJ Digital Med."},{"key":"B2","doi-asserted-by":"publisher","first-page":"558","DOI":"10.1016\/j.jhep.2019.10.003","article-title":"The gut-liver axis in liver disease: pathophysiological basis for therapy","volume":"72","author":"Albillos","year":"2020","journal-title":"J. Hepatol."},{"key":"B3","doi-asserted-by":"publisher","first-page":"e64","DOI":"10.1016\/S2589-7500(24)00224-3","article-title":"Tackling algorithmic bias and promoting transparency in health datasets: the STANDING together consensus recommendations","volume":"7","author":"Alderman","year":"2025","journal-title":"Lancet DIGIT. Health"},{"key":"B4","doi-asserted-by":"publisher","first-page":"1432","DOI":"10.1038\/s41591-021-01406-6","article-title":"Gut microbiota signatures are associated with toxicity to combined CTLA-4 and PD-1 blockade","volume":"27","author":"Andrews","year":"2021","journal-title":"Nat. Med."},{"key":"B5","doi-asserted-by":"publisher","first-page":"589","DOI":"10.1001\/jamainternmed.2023.1838","article-title":"Comparing physician and artificial intelligence chatbot responses to patient questions posted to a public social media forum","volume":"183","author":"Ayers","year":"2023","journal-title":"JAMA Intern. Med."},{"key":"B6","doi-asserted-by":"publisher","first-page":"97","DOI":"10.7326\/0003-4819-155-2-201107190-00005","article-title":"Low health literacy and health outcomes: an updated systematic review","volume":"155","author":"Berkman","year":"2011","journal-title":"Ann. Intern. Med."},{"key":"B7","doi-asserted-by":"publisher","first-page":"2071","DOI":"10.1111\/j.1572-0241.2007.01325.x","article-title":"A systematic review of patient inflammatory bowel disease information resources on the world wide web","volume":"102","author":"Bernard","year":"2007","journal-title":"Am. J. Gastroenterol."},{"key":"B8","doi-asserted-by":"publisher","first-page":"147","DOI":"10.1111\/liv.15340","article-title":"Immune checkpoint inhibitor-related cholangiopathy: novel clinicopathological description of a multi-centre cohort","volume":"43","author":"Berry","year":"2023","journal-title":"Liver Int. Off. J. Int. Assoc. Study Liver"},{"key":"B9","doi-asserted-by":"publisher","first-page":"531","DOI":"10.1038\/s41746-026-02987-7","article-title":"Embedded transparency in artificial intelligence: a prerequisite for equity and representation in AI-enabled clinical trials","volume":"9","author":"Blom","year":"2026","journal-title":"NPJ Digital Med."},{"key":"B10","doi-asserted-by":"publisher","first-page":"1714","DOI":"10.1200\/JCO.2017.77.6385","article-title":"Management of immune-related adverse events in patients treated with immune checkpoint inhibitor therapy: american society of clinical oncology clinical practice guideline","volume":"36","author":"Brahmer","year":"2018","journal-title":"J. Clin. Oncol. Off. J. Am. Soc Clin. Oncol."},{"key":"B11","doi-asserted-by":"publisher","first-page":"105","DOI":"10.1136\/jech.53.2.105","article-title":"DISCERN: an instrument for judging the quality of written consumer health information on treatment choices","volume":"53","author":"Charnock","year":"1999","journal-title":"J. Epidemiol. Community Health"},{"key":"B12","doi-asserted-by":"publisher","DOI":"10.1016\/j.artmed.2025.103346","article-title":"Learning health systems provide a glide path to safe landing for AI in health","volume":"173","author":"Curcin","year":"2026","journal-title":"Artif. Intell. Med."},{"key":"B13","doi-asserted-by":"publisher","first-page":"1384","DOI":"10.1053\/j.gastro.2020.08.063","article-title":"AGA clinical practice update on diagnosis and management of immune checkpoint inhibitor colitis and hepatitis: expert review","volume":"160","author":"Dougan","year":"2021","journal-title":"Gastroenterology"},{"key":"B14","doi-asserted-by":"publisher","DOI":"10.1016\/j.jjimei.2022.100067","article-title":"How do text characteristics impact user engagement in social media posts: modeling content readability, length, and hashtags number in Facebook","volume":"2","author":"Gkikas","year":"2022","journal-title":"Int. J. Inf. Manage. Data Insights"},{"key":"B15","doi-asserted-by":"publisher","first-page":"97","DOI":"10.1126\/science.aan4236","article-title":"Gut microbiome modulates response to anti-PD-1 immunotherapy in melanoma patients","volume":"359","author":"Gopalakrishnan","year":"2018","journal-title":"Science"},{"key":"B16","doi-asserted-by":"publisher","first-page":"1217","DOI":"10.1016\/j.annonc.2022.10.001","article-title":"Management of toxicities from immunotherapy: ESMO clinical practice guideline for diagnosis, treatment and follow-up","volume":"33","author":"Haanen","year":"2022","journal-title":"Ann. Oncol. Off. J. Eur. Soc Med. Oncol."},{"key":"B17","doi-asserted-by":"publisher","first-page":"e47","DOI":"10.1097\/CCM.0000000000006121","article-title":"Assessment of the readability of the online patient education materials of intensive and critical care societies","volume":"52","author":"Hanci","year":"2024","journal-title":"Crit. Care Med."},{"key":"B18","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pdig.0000198","article-title":"Performance of ChatGPT on USMLE: potential for AI-assisted medical education using large language models","volume":"2","author":"Kung","year":"2023","journal-title":"PLoS Digital Health"},{"key":"B19","doi-asserted-by":"publisher","first-page":"1233","DOI":"10.1056\/NEJMsr2214184","article-title":"Benefits, limits, and risks of GPT-4 as an AI chatbot for medicine","volume":"388","author":"Lee","year":"2023","journal-title":"N. Engl. J. Med."},{"key":"B20","doi-asserted-by":"publisher","first-page":"535","DOI":"10.1038\/s41591-022-01695-5","article-title":"Cross-cohort gut microbiome associations with immune checkpoint inhibitor response in advanced melanoma","volume":"28","author":"Lee","year":"2022","journal-title":"Nat. Med."},{"key":"B21","doi-asserted-by":"publisher","DOI":"10.1001\/jamanetworkopen.2022.46051","article-title":"Comparison of readability scores for written health information across formulas using automated vs manual measures","volume":"5","author":"Mac","year":"2022","journal-title":"JAMA Netw. Open"},{"key":"B22","doi-asserted-by":"publisher","first-page":"581","DOI":"10.1038\/s41746-025-01968-6","article-title":"Transforming healthcare delivery with conversational AI platforms","volume":"8","author":"Mahajan","year":"2025","journal-title":"NPJ Digital Med."},{"key":"B23","doi-asserted-by":"publisher","first-page":"104","DOI":"10.1126\/science.aao3290","article-title":"The commensal microbiome is associated with anti-PD-1 efficacy in metastatic melanoma patients","volume":"359","author":"Matson","year":"2018","journal-title":"Science"},{"key":"B24","doi-asserted-by":"publisher","first-page":"545","DOI":"10.1038\/s41591-022-01698-2","article-title":"Intestinal microbiota signatures of clinical response and immune-related adverse events in melanoma patients treated with anti-PD-1","volume":"28","author":"McCulloch","year":"2022","journal-title":"Nat. Med."},{"key":"B25","doi-asserted-by":"publisher","first-page":"165","DOI":"10.1111\/j.1369-7625.2004.00273.x","article-title":"Ensuring quality information for patients: development and preliminary validation of a new instrument to improve the quality of written health care information","volume":"7","author":"Moult","year":"2004","journal-title":"Health Expect.: Int. J. Public Particip. Health Care Health Policy"},{"key":"B26","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1016\/j.jksuci.2018.03.007","article-title":"Perusal of readability with focus on web content understandability","volume":"33","author":"Ojha","year":"2021","journal-title":"J. King Saud Univ. Comput. Inf. Sci."},{"key":"B27","doi-asserted-by":"publisher","DOI":"10.4081\/reumatismo.2022.1495","article-title":"Evaluating the readability, quality and reliability of online information on beh\u00e7et\u2019s disease","volume":"74","author":"\u00d6zduran","year":"2022","journal-title":"Reumatismo"},{"key":"B28","doi-asserted-by":"publisher","first-page":"158","DOI":"10.1056\/NEJMra1703481","article-title":"Immune-related adverse events associated with immune checkpoint blockade","volume":"378","author":"Postow","year":"2018","journal-title":"N. Engl. J. Med."},{"key":"B29","doi-asserted-by":"publisher","DOI":"10.1001\/jamanetworkopen.2024.0347","article-title":"Enhancing health care communication with large language models-the role, challenges, and future directions","volume":"7","author":"Raghu Subramanian","year":"2024","journal-title":"JAMA Netw. Open"},{"key":"B30","doi-asserted-by":"publisher","DOI":"10.1016\/j.jaut.2020.102514","article-title":"Best practices for detection, assessment and management of suspected immune-mediated liver injury caused by immune checkpoint inhibitors during drug development","volume":"114","author":"Regev","year":"2020","journal-title":"J. Autoimmun."},{"key":"B31","doi-asserted-by":"publisher","first-page":"1351","DOI":"10.1126\/science.aar4060","article-title":"Cancer immunotherapy using checkpoint blockade","volume":"359","author":"Ribas","year":"2018","journal-title":"Science"},{"key":"B32","doi-asserted-by":"publisher","first-page":"1244","DOI":"10.1001\/jama.1997.03540390074039","article-title":"Assessing, controlling, and assuring the quality of medical information on the internet: caveant lector et viewor--let the reader and viewer beware","volume":"277","author":"Silberg","year":"1997","journal-title":"JAMA"},{"key":"B33","doi-asserted-by":"publisher","first-page":"172","DOI":"10.1038\/s41586-023-06291-2","article-title":"Large language models encode clinical knowledge","volume":"620","author":"Singhal","year":"2023","journal-title":"Nature"},{"key":"B34","doi-asserted-by":"publisher","first-page":"112","DOI":"10.1038\/s41575-024-01019-7","article-title":"Immune-mediated liver injury from checkpoint inhibitors: mechanisms, clinical characteristics and management","volume":"22","author":"Triantafyllou","year":"2025","journal-title":"Nat. Rev. Gastroenterol. Hepatol."},{"key":"B35","doi-asserted-by":"publisher","first-page":"397","DOI":"10.1038\/s41575-018-0011-z","article-title":"The gut-liver axis and the intersection with the microbiome","volume":"15","author":"Tripathi","year":"2018","journal-title":"Nat. Rev. Gastroenterol. Hepatol."},{"key":"B36","doi-asserted-by":"publisher","first-page":"58","DOI":"10.1038\/s41746-024-01064-1","article-title":"To warrant clinical adoption AI models require a multi-faceted implementation evaluation","volume":"7","author":"van de Sande","year":"2024","journal-title":"NPJ Digital Med."},{"key":"B37","doi-asserted-by":"publisher","first-page":"58","DOI":"10.1038\/s41746-024-01390-4","article-title":"Towards evaluating and building versatile large language models for medicine","volume":"8","author":"Wu","year":"2025","journal-title":"NPJ Digital Med."},{"key":"B38","doi-asserted-by":"publisher","DOI":"10.1097\/MD.0000000000035543","article-title":"How readable and quality are online patient education materials about helicobacter pylori?: assessment of the readability, quality and reliability","volume":"102","author":"Yilmaz Hanci","year":"2023","journal-title":"Med. (Baltimore)"},{"key":"B39","doi-asserted-by":"publisher","first-page":"34","DOI":"10.1038\/s41746-025-02208-7","article-title":"Automating expert-level medical reasoning evaluation of large language models","volume":"9","author":"Zhou","year":"2025","journal-title":"NPJ Digital Med."}],"container-title":["Frontiers in Cellular and Infection Microbiology"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fcimb.2026.1955356\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,10,9]],"date-time":"2026-10-09T04:25:46Z","timestamp":1791519946000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fcimb.2026.1955356\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,10,9]]},"references-count":39,"alternative-id":["10.3389\/fcimb.2026.1955356"],"URL":"https:\/\/doi.org\/10.3389\/fcimb.2026.1955356","relation":{},"ISSN":["2235-2988"],"issn-type":[{"value":"2235-2988","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,10,9]]},"article-number":"1955356"}}