{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,30]],"date-time":"2026-06-30T19:01:29Z","timestamp":1782846089831,"version":"3.54.5"},"reference-count":50,"publisher":"JMIR Publications Inc.","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["JMIR Med Inform"],"abstract":"Abstract<\/jats:title>\n \n Background<\/jats:title>\n The prevalence of chronic gastritis is high, and if not intervened in a timely manner, it may eventually lead to gastric cancer. Managing chronic gastritis essentially requires comprehensive lifestyle changes. However, the current health care environment does not support continuous follow-up by professional health care providers, making self-management a key component of postdiagnosis care. Increasingly, researchers are exploring the use of large language models (LLMs) for patient management. However, LLMs have limitations, including hallucinations, limited knowledge scope, and lack of timeliness. Artificial intelligence (AI) agents may provide a more effective solution. Nevertheless, it remains uncertain whether AI agents can effectively support postdiagnosis self-management for patients with chronic gastritis.<\/jats:p>\n <\/jats:sec>\n \n Objective<\/jats:title>\n The purpose of this study was to explore the effectiveness of AI agents in the postdiagnosis management of patients with chronic gastritis from different perspectives.<\/jats:p>\n <\/jats:sec>\n \n Methods<\/jats:title>\n In this study, we developed an agent framework for the health management of patients with chronic gastritis based on LLMs in conjunction with retrieval-augmented generation and a search engine tool. We collected real questions from patients with chronic gastritis in clinical settings and tested the framework\u2019s performance across different difficulty levels and scenarios. We analyzed its safety and robustness and compared it with state-of-the-art models to comprehensively evaluate its effectiveness.<\/jats:p>\n <\/jats:sec>\n \n Results<\/jats:title>\n Using a dual-evaluation framework comprising automated metrics and expert manual assessments, our results demonstrated that AI agents substantially outperformed LLMs in addressing high-complexity questions (embedding average score: 82.849 for AI agents vs 77.825 for LLMs) and were particularly effective in clinical consultation tasks. Clinical evaluation of safety based on a 5-point Likert scale by physicians indicated that the safety of the agents was 4.98 (SD 0.15; 95% CI 4.96-4.99). After 30 repeated experiments, the mean absolute deviation of the AI agents in the embedding average score and BERTScore metrics were 0.0167 and 0.0387, respectively. Therefore, the safety and robustness analysis confirmed that the AI agents can produce safe, stable, and minimally variable responses. In addition, comparative results with those of advanced medical-domain LLMs (Baichuan-14B-M1 and MedGemma-27B) and general-domain LLMs (Qwen3-32B) also demonstrated that the AI agents in this study performed outstandingly in the field of chronic gastritis. Our findings underscore the superior reliability, interpretability, and practical applicability of AI agents over conventional LLMs in chronic gastritis management, offering a robust foundation for their broader adoption in health care settings.<\/jats:p>\n <\/jats:sec>\n \n Conclusions<\/jats:title>\n AI agents based on LLMs have high application value in the management of chronic gastritis. They can effectively guide patients with chronic diseases in addressing common issues, which may potentially reduce the workload of physicians and improve the quality of patient home care.<\/jats:p>\n <\/jats:sec>","DOI":"10.2196\/73857","type":"journal-article","created":{"date-parts":[[2025,11,14]],"date-time":"2025-11-14T14:35:40Z","timestamp":1763130940000},"page":"e73857-e73857","source":"Crossref","is-referenced-by-count":2,"title":["Enhancing Large Language Models With AI Agents for Chronic Gastritis Management: Comprehensive Comparative Study"],"prefix":"10.2196","volume":"13","author":[{"ORCID":"https:\/\/orcid.org\/0009-0006-6636-3586","authenticated-orcid":false,"given":"Shurui","family":"Wang","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5432-7741","authenticated-orcid":false,"given":"Qing","family":"Ye","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1010","published-online":{"date-parts":[[2025,11,13]]},"reference":[{"issue":"24","key":"R1","first-page":"6735","volume":"52","author":"Correa","journal-title":"Cancer Res"},{"issue":"3","key":"R2","doi-asserted-by":"publisher","first-page":"209","DOI":"10.3322\/caac.21660","article-title":"Global cancer statistics 2020: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries","volume":"71","author":"Sung","journal-title":"CA Cancer J Clin"},{"issue":"1","key":"R3","doi-asserted-by":"publisher","first-page":"108","DOI":"10.1186\/s12935-017-0475-x","article-title":"Volatile organic metabolites identify patients with gastric carcinoma, gastric ulcer, or gastritis and control patients","volume":"17","author":"Tong","journal-title":"Cancer Cell Int"},{"issue":"1","key":"R4","doi-asserted-by":"publisher","DOI":"10.1186\/1471-230X-14-149","article-title":"Health related quality of life in patients with chronic gastritis and peptic ulcer and factors with impact: a longitudinal study","volume":"14","author":"Wen","journal-title":"BMC Gastroenterol"},{"issue":"10","key":"R5","doi-asserted-by":"publisher","first-page":"3147","DOI":"10.21037\/apm-22-730","article-title":"The status and risk factors for anxiety\/depression in patients with atrophic chronic gastritis: a cross-sectional study","volume":"11","author":"Wu","journal-title":"Ann Palliat Med"},{"issue":"2","key":"R6","doi-asserted-by":"publisher","first-page":"70","DOI":"10.3126\/jmcjms.v10i2.47868","article-title":"Access to health services and health inequalities in remote and rural areas","volume":"10","author":"Osadolor","journal-title":"Janaki Med Coll J Med Sci"},{"issue":"13","key":"R7","doi-asserted-by":"publisher","first-page":"5444","DOI":"10.3390\/su12135444","article-title":"Examining trans-provincial diagnosis of rare diseases in China: the importance of healthcare resource distribution and patient mobility","volume":"12","author":"Yan","journal-title":"Sustainability"},{"issue":"4","key":"R8","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/1629096.1629101","article-title":"Cyberchondria: studies of the escalation of medical concerns in web search","volume":"27","author":"White","journal-title":"ACM Trans Inf Syst"},{"issue":"3","key":"R9","doi-asserted-by":"publisher","DOI":"10.2196\/jmir.1164","article-title":"Patients\u2019 attitudes toward electronic health information exchange: qualitative study","volume":"11","author":"Simon","journal-title":"J Med Internet Res"},{"issue":"9","key":"R10","doi-asserted-by":"publisher","DOI":"10.2196\/jmir.6877","article-title":"Putting the focus back on the patient: how privacy concerns affect personal health information sharing intentions","volume":"19","author":"Abdelhamid","journal-title":"J Med Internet Res"},{"key":"R11","unstructured":"Radford A Narasimhan K Salimans T Sutskever I . Improving language understanding by generative pre-training. OpenAI; 2018. URL: https:\/\/cdn.openai.com\/research-covers\/language-unsupervised\/language_understanding_paper.pdf [Accessed 09-10-2025]"},{"issue":"6","key":"R12","doi-asserted-by":"publisher","first-page":"501","DOI":"10.1136\/ejhpharm-2024-004245","article-title":"Investigating the capabilities of advanced large language models in generating patient instructions and patient educational material","volume":"32","author":"Sridharan","journal-title":"Eur J Hosp Pharm"},{"issue":"6","key":"R13","doi-asserted-by":"publisher","first-page":"589","DOI":"10.1001\/jamainternmed.2023.1838","article-title":"Comparing physician and artificial intelligence chatbot responses to patient questions posted to a public social media forum","volume":"183","author":"Ayers","journal-title":"JAMA Intern Med"},{"issue":"2","key":"R14","doi-asserted-by":"publisher","first-page":"115","DOI":"10.1017\/S0269888900008122","article-title":"Intelligent agents: theory and practice","volume":"10","author":"Wooldridge","journal-title":"Knowl Eng Rev"},{"key":"R15","unstructured":"Gosmar D Dahl DA . Hallucination mitigation using agentic AI natural language-based frameworks. arXiv. Preprint posted online on Jan 19, 2025. [doi: 10.48550\/arXiv.2501.13946]"},{"key":"R16","doi-asserted-by":"crossref","unstructured":"Wang J Duan Z . Controlling large language model hallucination based on agent ai with langgraph. Cambridge Open Engage. Preprint posted online on Jan 13, 2025. [doi: 10.33774\/coe-2025-xkwl5]","DOI":"10.33774\/coe-2025-xkwl5"},{"key":"R17","doi-asserted-by":"crossref","unstructured":"Wallat J Jatowt A Anand A . Temporal blind spots in large language models. In: Ang\u00e9lica L Lattanzi S Medina AM , editors. WSDM \u201924: Proceedings of the 17th ACM International Conference on Web Search and Data Mining. Association for Computing Machinery; 2024. [doi: 10.1145\/3616855.3635818] ISBN: 9798400703713","DOI":"10.1145\/3616855.3635818"},{"key":"R18","doi-asserted-by":"crossref","unstructured":"Park JS O\u2019Brien J Cai CJ Morris MR Liang P Bernstein MS . Generative agents: interactive simulacra of human behavior. In: Riche NH Han J Steimle J Riche NH , editors. UIST \u201923: Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology. Association for Computing Machinery; 2023. [doi: 10.1145\/3586183.3606763] ISBN: 9798400701320","DOI":"10.1145\/3586183.3606763"},{"key":"R19","unstructured":"Liu R Yang R Jia C Zhang G Zhou D Dai AM et al. Training socially aligned language models on simulated social interactions. arXiv. Preprint posted online on Oct 28, 2023. [doi: 10.48550\/arXiv.2305.16960]"},{"key":"R20","unstructured":"Sumers TR Yao S Narasimhan K Griffiths TL . Cognitive architectures for language agents. arXiv. Preprint posted online on Mar 15, 2024. [doi: 10.48550\/arXiv.2309.02427]"},{"key":"R21","doi-asserted-by":"crossref","unstructured":"Shi W Xu R Zhuang Y et al. EHRAgent: code empowers large language models for few-shot complex tabular reasoning on electronic health records. Presented at: Conference on Empirical Methods in Natural Language Processing; Nov 12-16, 2024; Miami, FL. [doi: 10.18653\/v1\/2024.emnlp-main.1245]","DOI":"10.18653\/v1\/2024.emnlp-main.1245"},{"key":"R22","doi-asserted-by":"crossref","unstructured":"Zhang W Tang K Wu H et al. Agent-pro: learning to evolve via policy-level reflection and optimization. Presented at: Annual Meeting of the Association for Computational Linguistics; Aug 11-16, 2024; Bangkok, Thailand. [doi: 10.18653\/v1\/2024.acl-long.292]","DOI":"10.18653\/v1\/2024.acl-long.292"},{"key":"R23","doi-asserted-by":"crossref","unstructured":"Zhao J Zu C Hao X et al. LONGAGENT: achieving question answering for 128k-token-long documents through multi-agent collaboration. In: Chen YN Bansal M Chen YN , editors. Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics; 2024. [doi: 10.18653\/v1\/2024.emnlp-main.912]","DOI":"10.18653\/v1\/2024.emnlp-main.912"},{"key":"R24","doi-asserted-by":"crossref","unstructured":"Lu Y Bartolo M Moore A Riedel S Stenetorp P . Fantastically ordered prompts and where to find them: overcoming few-shot prompt order sensitivity. Presented at: Annual Meeting of the Association for Computational Linguistics; May 22-27, 2022; Dublin, Ireland. [doi: 10.18653\/v1\/2022.acl-long.556]","DOI":"10.18653\/v1\/2022.acl-long.556"},{"key":"R25","unstructured":"Chu K Chen YP Nakayama H . A better LLM evaluator for text generation: the impact of prompt output sequencing and optimization. arXiv. Preprint posted online on Jun 14, 2024. [doi: 10.48550\/arXiv.2406.09972]"},{"key":"R26","unstructured":"Zhao S Yang Y Wang Z He Z Qiu LK Qiu L . Retrieval augmented generation (RAG) and beyond: a comprehensive survey on how to make your llms use external data more wisely. arXiv. Preprint posted online on Sep 23, 2024. [doi: 10.48550\/arXiv.2409.14924]"},{"issue":"8","key":"R27","doi-asserted-by":"publisher","first-page":"298","DOI":"10.3390\/fi16080298","article-title":"A survey on the use of large language models (LLMs) in fake news","volume":"16","author":"Papageorgiou","journal-title":"Future Internet"},{"key":"R28","unstructured":"Yang A Yang B Zhang B Hui B Zheng B Yu B et al. Qwen2.5 technical report. arXiv. Preprint posted online on Jan 3, 2025. [doi: 10.48550\/arXiv.2412.15115]"},{"key":"R29","unstructured":"Yang A Yang B Hui B Zheng B Yu B Zhou C et al. Qwen2 technical report. arXiv. Preprint posted online on Sep 10, 2024. [doi: 10.48550\/arXiv.2407.10671]"},{"key":"R30","unstructured":"Lewis P Perez E Piktus A Petroni F Karpukhin V Goyal N et al. Retrieval-augmented generation for knowledge-intensive NLP tasks. arXiv. Preprint posted online on Apr 12, 2021. [doi: 10.48550\/arXiv.2005.11401]"},{"issue":"1","key":"R31","doi-asserted-by":"publisher","DOI":"10.1056\/AIra2400380","article-title":"RAG in health care: a novel framework for improving communication and decision-making by addressing LLM limitations","volume":"2","author":"Ng","journal-title":"NEJM AI"},{"key":"R32","doi-asserted-by":"crossref","unstructured":"Agirre E Alfonseca E Hall K Kravalova J Pa\u015fca M Soroa A . A study on similarity and relatedness using distributional and wordnet-based approaches. In: Ostendorf M , editor. NAACL \u201909: Proceedings of Human Language Technologies: The 2009 Annual Conference of the North American Chapter of the Association for Computational Linguistics. Association for Computational Linguistics; 2009. [doi: 10.3115\/1620754.1620758] ISBN: 9781932432411","DOI":"10.3115\/1620754.1620758"},{"issue":"12","key":"R33","doi-asserted-by":"publisher","first-page":"e855","DOI":"10.1016\/S2589-7500(23)00202-9","article-title":"Using fine-tuned large language models to parse clinical notes in musculoskeletal pain disorders","volume":"5","author":"Vaid","journal-title":"Lancet Digit Health"},{"issue":"4","key":"R34","doi-asserted-by":"publisher","first-page":"1134","DOI":"10.1038\/s41591-024-02855-5","article-title":"Adapted large language models can outperform medical experts in clinical text summarization","volume":"30","author":"Van Veen","journal-title":"Nat Med"},{"issue":"1","key":"R35","doi-asserted-by":"publisher","DOI":"10.1038\/s41746-024-01258-7","article-title":"A framework for human evaluation of large language models in healthcare derived from literature review","volume":"7","author":"Tam","journal-title":"NPJ Digit Med"},{"issue":"1","key":"R36","doi-asserted-by":"publisher","first-page":"83","DOI":"10.13458\/j.cnki.flatt.004997","article-title":"AlphaReadabilityChinese: a tool for the measurement of readability in Chinese texts and its applications","volume":"2024","author":"Lei","journal-title":"Foreign Lang Teach"},{"key":"R37","unstructured":"Lee BW Jang YS Lee JJ . Pushing on text readability assessment: a transformer meets handcrafted linguistic features. arXiv. Preprint posted online on Jan 16, 2024. [doi: 10.48550\/arXiv.2109.12258]"},{"issue":"7972","key":"R38","doi-asserted-by":"publisher","first-page":"172","DOI":"10.1038\/s41586-023-06291-2","article-title":"Large language models encode clinical knowledge","volume":"620","author":"Singhal","journal-title":"Nature New Biol"},{"issue":"1","key":"R39","doi-asserted-by":"publisher","DOI":"10.1007\/s10916-023-01925-4","article-title":"Evaluating the feasibility of ChatGPT in healthcare: an analysis of multiple clinical and research scenarios","volume":"47","author":"Cascella","journal-title":"J Med Syst"},{"issue":"8","key":"R40","doi-asserted-by":"publisher","first-page":"1930","DOI":"10.1038\/s41591-023-02448-8","article-title":"Large language models in medicine","volume":"29","author":"Thirunavukarasu","journal-title":"Nat Med"},{"key":"R41","unstructured":"Shi F Chen X Misra K Scales N Dohan D Chi EH et al. Large language models can be easily distracted by irrelevant context. In: Krause A Brunskill E Cho K Engelhardt B Sabato S , editors. ICML\u201923: Proceedings of the 40th International Conference on Machine Learning. JMLR.org; 2023. URL: https:\/\/dl.acm.org\/doi\/10.5555\/3618408.3619699 [Accessed 25-05-2025]"},{"key":"R42","unstructured":"Wang B Zhao H Zhou H Song L Xu M Cheng W et al. Baichuan-M1: pushing the medical capability of large language models. arXiv. Preprint posted online on Mar 5, 2025. [doi: 10.48550\/arXiv.2502.12671]"},{"key":"R43","unstructured":"Sellergren A Kazemzadeh S Jaroensri T Kiraly A Traverse M Kohlberger T et al. MedGemma technical report. arXiv. Preprint posted online on Jul 12, 2025. [doi: 10.48550\/arXiv.2507.05201]"},{"key":"R44","unstructured":"Yang A Li A Yang B Zhang B Hui B Zheng B et al. Qwen3 technical report. arXiv. Preprint posted online on May 14, 2025. [doi: 10.48550\/arXiv.2505.09388]"},{"key":"R45","unstructured":"Tyson A Pasquini G Spencer A Funk C . 60% of Americans would be uncomfortable with provider relying on AI in their own health care. Pew Research Center. 2025. URL: https:\/\/www.pewresearch.org\/science\/2023\/02\/22\/60-of-americans-would-be-uncomfortable-with-provider-relying-on-ai-in-their-own-health-care\/ [Accessed 30-07-2025]"},{"issue":"3","key":"R46","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3641289","article-title":"A survey on evaluation of large language models","volume":"15","author":"Chang","journal-title":"ACM Trans Intell Syst Technol"},{"key":"R47","unstructured":"Ji J Liu M Dai J Pan X Zhang C Bian C et al. BeaverTails: towards improved safety alignment of LLM via a human-preference dataset. arXiv. Preprint posted online on Nov 7, 2023. [doi: 10.48550\/arXiv.2307.04657]"},{"key":"R48","doi-asserted-by":"crossref","unstructured":"Tan TF Elangovan K Ong JCL et al. A proposed s.c.o.r.e. evaluation framework for large language models \u2013 safety, consensus & context, objectivity, reproducibility and explainability. arXiv. Preprint posted online on Jul 10, 2024. [doi: 10.2139\/ssrn.5029562]","DOI":"10.2139\/ssrn.5029562"},{"key":"R49","doi-asserted-by":"crossref","unstructured":"Zhang H Chen J Jiang F et al. HuatuoGPT, towards taming language model to be a doctor. Presented at: Findings of the Association for Computational Linguistics: EMNLP 2023; Dec 6-10, 2023. [doi: 10.18653\/v1\/2023.findings-emnlp.725]","DOI":"10.18653\/v1\/2023.findings-emnlp.725"},{"key":"R50","unstructured":"JMI-codes. GitHub. URL: https:\/\/github.com\/472404420\/JMI-codes [Accessed 30-10-2025]"}],"container-title":["JMIR Medical Informatics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/medinform.jmir.org\/2025\/1\/e73857","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,6,30]],"date-time":"2026-06-30T18:04:46Z","timestamp":1782842686000},"score":1,"resource":{"primary":{"URL":"https:\/\/medinform.jmir.org\/2025\/1\/e73857"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,11,13]]},"references-count":50,"URL":"https:\/\/doi.org\/10.2196\/73857","relation":{},"ISSN":["2291-9694"],"issn-type":[{"value":"2291-9694","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,11,13]]},"article-number":"v13i24e73857"}}