{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,9]],"date-time":"2026-09-09T00:08:51Z","timestamp":1788912531595,"version":"build-2803163510"},"reference-count":18,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2026,6,25]],"date-time":"2026-06-25T00:00:00Z","timestamp":1782345600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2026,9,9]],"date-time":"2026-09-09T00:00:00Z","timestamp":1788912000000},"content-version":"vor","delay-in-days":76,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["BMC Med Educ"],"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:sec>\n                    <jats:title>Background<\/jats:title>\n                    <jats:p>Medical education relies on experience to develop clinical reasoning skills, yet patient access is often limited. Large language models (LLMs) offer an accessible alternative for simulating patient encounters. We aimed to develop and evaluate an LLM-based clinical simulation system for gastroenterology that simulates entire clinical encounters in natural language, from history-taking through to treatment and follow-up planning.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Methods<\/jats:title>\n                    <jats:p>The study was conducted in three phases. First, a system comprising an LLM-based clinical case generator, an LLM-based patient simulator, and a web chat interface was developed and iteratively refined. Second, two gastroenterology fellows performed 20 clinical case simulations each as initial testers, providing feedback that guided further system adjustments. Third, six independent gastroenterology fellows, blinded to the development process, evaluated the final system by completing the same 20 clinical simulations. All evaluators rated the system using a 5-point Likert scale across four domains: case presentation accuracy, adaptability to user interactions, realism, and educational value.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>\n                      A total of 160 evaluations were collected across 20 cases and 8 raters. At final evaluation, all domains scored highly: educational value (mean 4.57\u2009\u00b1\u20090.63), accuracy (4.42\u2009\u00b1\u20090.63), adaptability (4.33\u2009\u00b1\u20090.75), and realism (4.32\u2009\u00b1\u20090.81). Scores differed significantly across domains (\n                      <jats:italic>p<\/jats:italic>\n                      \u2009&lt;\u20090.01), with educational value rated highest. Evaluation panellists rated all domains significantly higher than initial testers (all\n                      <jats:italic>p<\/jats:italic>\n                      \u2009&lt;\u20090.01) at an exploratory comparison.\n                    <\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusions<\/jats:title>\n                    <jats:p>The LLM-based clinical simulator demonstrated high accuracy, realism, adaptability, and educational value for gastroenterology case simulation in this pilot evaluation, and may represent an accessible and cost-effective tool for clinical reasoning training.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.1186\/s12909-026-09709-3","type":"journal-article","created":{"date-parts":[[2026,6,25]],"date-time":"2026-06-25T03:33:59Z","timestamp":1782358439000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Development and pilot evaluation of a large language model-based clinical simulation system for medical education in gastroenterology"],"prefix":"10.1186","volume":"26","author":[{"given":"Stiliano","family":"Maimaris","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Gianmarco","family":"Tedino","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Giusi Aurora","family":"Memoli","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Carlotta","family":"Crisciotti","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Alessandra","family":"Albrizio","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Camilla","family":"Sammartino","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Erica","family":"Bartolotta","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Giulio","family":"Massetti","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Giulia","family":"Mantica","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Paolo","family":"Minerba","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Federico","family":"Biagi","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Annalisa","family":"Schiepatti","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2026,6,25]]},"reference":[{"key":"9709_CR1","doi-asserted-by":"publisher","first-page":"6120","DOI":"10.5688\/ajpe6120","volume":"81","author":"C Gillette","year":"2017","unstructured":"Gillette C, Stanton RB, Rockich-Winston N, Rudolph M, Anderson HG Jr. Cost-Effectiveness of Using Standardized Patients to Assess Student-Pharmacist Communication Skills. Am J Pharm Educ. 2017;81:6120.","journal-title":"Am J Pharm Educ"},{"key":"9709_CR2","doi-asserted-by":"publisher","first-page":"443","DOI":"10.1097\/00001888-199306000-00002","volume":"68","author":"HS Barrows","year":"1993","unstructured":"Barrows HS. An overview of the uses of standardized patients for teaching and evaluating clinical skills. AAMC Acad Med. 1993;68:443\u201351. discussion 451-3.","journal-title":"AAMC Acad Med"},{"key":"9709_CR3","unstructured":"Bubeck S, Chandrasekaran V, Eldan R, Gehrke J, Horvitz E, Kamar E et al. Sparks of artificial general intelligence: early experiments with GPT-4. arXiv [Preprint] 2023. Available from: https:\/\/arxiv.org\/abs\/2303.12712.\u00a0Accessed 7 May 2026."},{"key":"9709_CR4","doi-asserted-by":"publisher","first-page":"e0000198","DOI":"10.1371\/journal.pdig.0000198","volume":"2","author":"TH Kung","year":"2023","unstructured":"Kung TH, Cheatham M, Medenilla A, Sillos C, De Leon L, Elepa\u00f1o C, Madriaga M, Aggabao R, Diaz-Candido G, Maningo J, Tseng V. Performance of ChatGPT on USMLE: Potential for AI-assisted medical education using large language models. PLOS Digit Health. 2023;2:e0000198.","journal-title":"PLOS Digit Health"},{"key":"9709_CR5","doi-asserted-by":"publisher","first-page":"1391","DOI":"10.1186\/s12909-024-06399-7","volume":"24","author":"E Br\u00fcgge","year":"2024","unstructured":"Br\u00fcgge E, Ricchizzi S, Arenbeck M, Keller MN, Schur L, Stummer W, Holling M, Lu MH, Darici D. Large language models improve clinical decision making of medical students through patient simulation and structured feedback: a randomized controlled trial. BMC Med Educ. 2024;24:1391.","journal-title":"BMC Med Educ"},{"key":"9709_CR6","doi-asserted-by":"publisher","first-page":"60","DOI":"10.1186\/s12911-022-01797-7","volume":"22","author":"M Wang","year":"2022","unstructured":"Wang M, Sun Z, Jia M, Wang Y, Wang H, Zhu X, Chen L, Ji H. Intelligent virtual case learning system based on real medical records and natural language processing. BMC Med Inf Decis Mak. 2022;22:60.","journal-title":"BMC Med Inf Decis Mak"},{"key":"9709_CR7","doi-asserted-by":"publisher","first-page":"e24073","DOI":"10.2196\/24073","volume":"9","author":"R Furlan","year":"2021","unstructured":"Furlan R, Gatti M, Men\u00e8 R, Shiffer D, Marchiori C, Giaj Levra A, Saturnino V, Brunetta E, Dipaola F. A Natural Language Processing-Based Virtual Patient Simulator and Intelligent Tutoring System for the Clinical Diagnostic Process: Simulator Development and Case Study. JMIR Med Inf. 2021;9:e24073.","journal-title":"JMIR Med Inf"},{"key":"9709_CR8","doi-asserted-by":"publisher","first-page":"1745","DOI":"10.1080\/0142159X.2025.2501252","volume":"47","author":"M Moser","year":"2025","unstructured":"Moser M, Posel N, Ganescu O, Fleiszer D. Twelve tips: Using generative AI to create and optimize content for virtual patient simulations. Med Teach. 2025;47:1745\u201351.","journal-title":"Med Teach"},{"key":"9709_CR9","doi-asserted-by":"publisher","first-page":"259","DOI":"10.1186\/s12909-025-06756-0","volume":"25","author":"K R\u00e4del-Ablass","year":"2025","unstructured":"R\u00e4del-Ablass K, Schliz K, Schlick C, Meindl B, Pahr-Hosbach S, Schwendemann H, Rupp S, Roddewig M, Miersch C. Teaching opportunities for anamnesis interviews through AI based teaching role plays: a survey with online learning students from health study programs. BMC Med Educ. 2025;25:259.","journal-title":"BMC Med Educ"},{"key":"9709_CR10","doi-asserted-by":"publisher","first-page":"e59435","DOI":"10.2196\/59435","volume":"27","author":"C Wang","year":"2025","unstructured":"Wang C, Li S, Lin N, Zhang X, Han Y, Wang X, Liu D, Tan X, Pu D, Li K, Qian G, Yin R. Application of Large Language Models in Medical Training Evaluation-Using ChatGPT as a Standardized Patient: Multimetric Assessment. J Med Internet Res. 2025;27:e59435.","journal-title":"J Med Internet Res"},{"key":"9709_CR11","doi-asserted-by":"publisher","first-page":"40","DOI":"10.1080\/0142159X.2024.2376879","volume":"47","author":"DA Cook","year":"2025","unstructured":"Cook DA. Creating virtual patients using large language models: scalable, global, and low cost. Med Teach. 2025;47:40\u20132.","journal-title":"Med Teach"},{"key":"9709_CR12","doi-asserted-by":"publisher","first-page":"e66478","DOI":"10.2196\/66478","volume":"9","author":"R Scherr","year":"2025","unstructured":"Scherr R, Spina A, Dao A, Andalib S, Halaseh FF, Blair S, Wiechmann W, Rivera R. Novel Evaluation Metric and Quantified Performance of ChatGPT-4 Patient Management Simulations for Early Clinical Education: Experimental Study. JMIR Form Res. 2025;9:e66478.","journal-title":"JMIR Form Res"},{"key":"9709_CR13","unstructured":"Radford A, Kim JW, Xu T, Brockman G, McLeavey C, Sutskever I. Robust speech recognition via large-scale weak supervision. arXiv [Preprint]. 2022. Available from: https:\/\/arxiv.org\/abs\/2212.04356.\u00a0Accessed 7 May 2026."},{"key":"9709_CR14","unstructured":"Artificial Analysis. Gemini 2.5 Pro Intelligence, Performance & Price Analysis. Available from: https:\/\/artificialanalysis.ai\/models\/gemini-2-5-pro\/.\u00a0[Cited 20 Feb 2026]."},{"key":"9709_CR15","unstructured":"Artificial Analysis. Gemini 2.0 Flash (Feb \u201825) Intelligence, Performance & Price Analysis. Available from: https:\/\/artificialanalysis.ai\/models\/gemini-2-0-flash\/.\u00a0[Cited 20 Feb 2026]."},{"key":"9709_CR16","unstructured":"Artificial Analysis. LLM Leaderboard - Comparison of over 100 AI models from OpenAI, Google, DeepSeek & others. Available from: https:\/\/artificialanalysis.ai\/leaderboards\/models?deprecation=all. [Cited 20 Feb 2026]."},{"key":"9709_CR17","doi-asserted-by":"publisher","first-page":"100900","DOI":"10.1016\/j.landig.2025.100900","volume":"7","author":"Y Ning","year":"2025","unstructured":"Ning Y, Ong JCL, Cheng H, Wang H, Ting DSW, Tham YC, Wong TY, Liu N. How can artificial intelligence transform the training of medical students and physicians? Lancet Digit Health. 2025;7:100900.","journal-title":"Lancet Digit Health"},{"key":"9709_CR18","doi-asserted-by":"publisher","first-page":"613","DOI":"10.2147\/AMEP.S312373","volume":"12","author":"PJ Hershberger","year":"2021","unstructured":"Hershberger PJ, Pei Y, Bricker DA, Crawford TN, Shivakumar A, Vasoya M, Medaramitta R, Rechtin M, Bositty A, Wilson JF. Advancing Motivational Interviewing Training with Artificial Intelligence: ReadMI. Adv Med Educ Pract. 2021;12:613\u20138.","journal-title":"Adv Med Educ Pract"}],"container-title":["BMC Medical Education"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/article\/10.1186\/s12909-026-09709-3","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s12909-026-09709-3.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s12909-026-09709-3.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,9,9]],"date-time":"2026-09-09T00:02:03Z","timestamp":1788912123000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1186\/s12909-026-09709-3"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,6,25]]},"references-count":18,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2026,12]]}},"alternative-id":["9709"],"URL":"https:\/\/doi.org\/10.1186\/s12909-026-09709-3","relation":{},"ISSN":["1472-6920"],"issn-type":[{"value":"1472-6920","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,6,25]]},"assertion":[{"value":"7 May 2026","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"11 June 2026","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"25 June 2026","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"In this study no patients were involved and no patient data, or biological samples were used. All participating clinician evaluators provided informed consent and are co-authors of this manuscript. All clinical cases were entirely fictional and generated by artificial intelligence. The evaluation consisted of gastroenterology fellows rating the performance of a software tool. Under Italian legislation the oversight of ethics committees pertains to biomedical research including clinical trials of medicinal products for human use, clinical investigations of medical devices, and observational pharmacological and non-pharmacological clinical studies, none of which apply to the present study. Ethics committee approval was therefore not applicable under Italian regulations. The study was conducted in accordance with the Declaration of Helsinki where applicable.","order":1,"name":"Ethics","label":"Ethics approval and consent to participate","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"Not applicable.","order":2,"name":"Ethics","label":"Consent for publication","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare no competing interests.","order":3,"name":"Ethics","label":"Competing interests","group":{"name":"EthicsHeading","label":"Declarations"}}],"article-number":"1442"}}