{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,15]],"date-time":"2026-09-15T07:14:20Z","timestamp":1789456460180,"version":"build-2803163510"},"reference-count":29,"publisher":"Walter de Gruyter GmbH","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"abstract":"Abstract<\/jats:title>\n \n Objectives<\/jats:title>\n Clinical reasoning develops through repeated, deliberate practice, yet clinical and simulation environments are often limited by continuity, feedback, and scalability constraints. Large language models (LLMs) may address this by generating virtual clinical encounters accessible without live instructors or standardized patients but validity evidence remains limited. This study explored the performance of MAESSCR (Multi-Agent Educational Scenario Simulator for Clinical Reasoning), a platform where multiple LLM-based agents, each assigned a distinct role (i.e.\u00a0patient, physical exam, diagnostic testing) interact with learners through a text-based encounter.<\/jats:p>\n <\/jats:sec>\n \n Methods<\/jats:title>\n In Fall of 2024, 175\u202fsecond-year medical students completed three MAESSCR clinical encounters as coursework. Six clinician-educators developed a rating tool to evaluate 120 transcripts (40 randomly sampled per case) using a dichotomous (yes\/no) scale across four domains: (1) realism of agent responses, (2) adherence to scripted case details, (3) platform functionality, and (4) interference with students\u2019 independent reasoning through clinical findings. Qualitative narrative review supplemented binary ratings.<\/jats:p>\n <\/jats:sec>\n \n Results<\/jats:title>\n MAESSCR followed scripted details in 92\u202f% (110\/120) of transcripts. Diagnosis-changing information occurred in 2.5\u202f% (3\/120). Unrealistic patient portrayal appeared in 12\u202f% (14\/120) of encounters, and technical issues in 17\u202f% (20\/120). AI agents most often interfered with student\u2019s clinical reasoning by interpreting findings before the students had the opportunity. This occurred in 28\u202f% (34\/120) of encounters due to history\/physical exam agents and 59\u202f% (71\/120) because of diagnostics\/management agents. Most disruptions were minor and unlikely to compromise the overall encounter.<\/jats:p>\n <\/jats:sec>\n \n Conclusions<\/jats:title>\n Multi-agent, LLM-based simulations offer a scalable approach to deliberate practice of clinical reasoning. Educational value depends on role stability, contextual fidelity, and learners\u2019 opportunity to interpret clinical information independently. Specific design considerations are essential to ensure AI-generated simulations support, rather than disrupt, the clinical reasoning process.<\/jats:p>\n <\/jats:sec>","DOI":"10.1515\/dx-2026-0069","type":"journal-article","created":{"date-parts":[[2026,9,15]],"date-time":"2026-09-15T06:56:05Z","timestamp":1789455365000},"source":"Crossref","is-referenced-by-count":0,"title":["Early evidence for a multi-agent AI simulator for clinical reasoning practice: performance, consistency, and challenges"],"prefix":"10.1515","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-6400-1745","authenticated-orcid":false,"given":"Matthew E.","family":"Kelleher","sequence":"first","affiliation":[{"name":"Department of Pediatrics and Internal Medicine , Cincinnati Children\u2019s Hospital Medical Center and University of Cincinnati College of Medicine , Cincinnati , OH , USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Christine Y.","family":"Zhou","sequence":"additional","affiliation":[{"name":"Division of Pulmonary, Critical Care, and Sleep Medicine, Department of Internal Medicine , University of Cincinnati , Cincinnati , OH , USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Danielle E.","family":"Weber","sequence":"additional","affiliation":[{"name":"Department of Pediatrics and Internal Medicine , Cincinnati Children\u2019s Hospital Medical Center and University of Cincinnati College of Medicine , Cincinnati , OH , USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0001-3696-5395","authenticated-orcid":false,"given":"Seth","family":"Overla","sequence":"additional","affiliation":[{"name":"Office of Medical Education , University of Cincinnati College of Medicine , Cincinnati , OH , USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8199-9076","authenticated-orcid":false,"given":"James","family":"Bowen","sequence":"additional","affiliation":[{"name":"Department of Pediatrics , University of Cincinnati College of Medicine , Cincinnati , OH , USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jose","family":"Generoso","sequence":"additional","affiliation":[{"name":"Department of Pediatrics , University of Cincinnati College of Medicine , Cincinnati , OH , USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Weibing","family":"Zheng","sequence":"additional","affiliation":[{"name":"University of Cincinnati College of Education, Criminal Justice, and Human Services, and School of Information Technology , Cincinnati , OH , USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8327-8002","authenticated-orcid":false,"given":"Sally A.","family":"Santen","sequence":"additional","affiliation":[{"name":"University of Cincinnati , Cincinnati , OH , USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4567-1313","authenticated-orcid":false,"given":"Laurah","family":"Turner","sequence":"additional","affiliation":[{"name":"Department of Medical Education, and Biostatistics, Health Informatics and Data Sciences , University of Cincinnati College of Medicine , Cincinnati , OH , USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"374","published-online":{"date-parts":[[2026,9,16]]},"reference":[{"key":"2026091506555459401_j_dx-2026-0069_ref_001","doi-asserted-by":"crossref","unstructured":"Schuwirth, LWT, Durning, SJ, King, SM. Assessment of clinical reasoning: three evolutions of thought. Diagnosis (Berl) 2020;7:191\u20136. https:\/\/doi.org\/10.1515\/dx-2019-0096.","DOI":"10.1515\/dx-2019-0096"},{"key":"2026091506555459401_j_dx-2026-0069_ref_002","doi-asserted-by":"crossref","unstructured":"Daniel, M, Rencic, J, Durning, SJ, Holmboe, E, Santen, SA, Lang, V, et al.. Clinical reasoning assessment methods: a scoping review and practical guidance. Acad Med 2019;94:902\u201312. https:\/\/doi.org\/10.1097\/ACM.0000000000002618.","DOI":"10.1097\/ACM.0000000000002618"},{"key":"2026091506555459401_j_dx-2026-0069_ref_003","doi-asserted-by":"crossref","unstructured":"Sinha, P, Pischel, L, Sofair, AN. Improving diagnosis by feedback and deliberate practice: one-on-one coaching for diagnostic maturation. Diagnosis (Berl) 2021;8:157\u201360. https:\/\/doi.org\/10.1515\/dx-2020-0129.","DOI":"10.1515\/dx-2020-0129"},{"key":"2026091506555459401_j_dx-2026-0069_ref_004","doi-asserted-by":"crossref","unstructured":"Sellberg, M, Palmgren, PJ, Moller, R. A cross-sectional study of clinical learning environments across four undergraduate programs using the undergraduate clinical education environment measure. BMC Med Educ 2021;21:258. https:\/\/doi.org\/10.1186\/s12909-021-02687-8.","DOI":"10.1186\/s12909-021-02687-8"},{"key":"2026091506555459401_j_dx-2026-0069_ref_005","doi-asserted-by":"crossref","unstructured":"Warm, EJ, Desai, SS, Bowen, JL. Navigating the discontinuity crisis in medical education. N\u00a0Engl J Med 2025;392:2447\u201357. https:\/\/doi.org\/10.1056\/NEJMra2408679.","DOI":"10.1056\/NEJMra2408679"},{"key":"2026091506555459401_j_dx-2026-0069_ref_006","doi-asserted-by":"crossref","unstructured":"Wagner, FL, Sudacka, M, Kononowicz, AA, Elven, M, Durning, SJ, Hege, I, et al.. Current status and ongoing needs for the teaching and assessment of clinical reasoning\u00a0\u2013\u00a0an international mixed-methods study from the students; and teachers; perspective. BMC Med Educ 2024;24:622. https:\/\/doi.org\/10.1186\/s12909-024-05518-8.","DOI":"10.1186\/s12909-024-05518-8"},{"key":"2026091506555459401_j_dx-2026-0069_ref_007","doi-asserted-by":"crossref","unstructured":"Ericsson, KA. Acquisition and maintenance of medical expertise: a perspective from the expert-performance approach with deliberate practice. Acad Med 2015;90:1471\u201386. https:\/\/doi.org\/10.1097\/ACM.0000000000000939.","DOI":"10.1097\/ACM.0000000000000939"},{"key":"2026091506555459401_j_dx-2026-0069_ref_008","doi-asserted-by":"crossref","unstructured":"Schaye, V, Miller, L, Kudlowitz, D, Chun, J, Burk-Rafel, J, Cocks, P, et al.. Development of a clinical reasoning documentation assessment tool for resident and fellow admission notes: a shared mental model for feedback. J\u00a0Gen Intern Med 2022;37:507\u201312. https:\/\/doi.org\/10.1007\/s11606-021-06805-6.","DOI":"10.1007\/s11606-021-06805-6"},{"key":"2026091506555459401_j_dx-2026-0069_ref_009","doi-asserted-by":"crossref","unstructured":"Staal, J, Waechter, J, Allen, J, Lee, CH, Zwaan, L. Deliberate practice of diagnostic clinical reasoning reveals low performance and improvement of diagnostic justification in pre-clerkship students. BMC Med Educ 2023;23:684. https:\/\/doi.org\/10.1186\/s12909-023-04541-5.","DOI":"10.1186\/s12909-023-04541-5"},{"key":"2026091506555459401_j_dx-2026-0069_ref_010","doi-asserted-by":"crossref","unstructured":"McGaghie, WC, Issenberg, SB, Cohen, ER, Barsuk, JH, Wayne, DB. Does simulation-based medical education with deliberate practice yield better results than traditional clinical education? A meta-analytic comparative review of the evidence. Acad Med 2011;86:706\u201311. https:\/\/doi.org\/10.1097\/ACM.0b013e318217e119.","DOI":"10.1097\/ACM.0b013e318217e119"},{"key":"2026091506555459401_j_dx-2026-0069_ref_011","doi-asserted-by":"crossref","unstructured":"Elendu, C, Amaechi, DC, Okatta, AU, Amaechi, EC, Elendu, TC, Ezeh, CP, et al.. The impact of simulation-based training in medical education: a review. Medicine (Baltim) 2024;103:e38813. https:\/\/doi.org\/10.1097\/MD.0000000000038813.","DOI":"10.1097\/MD.0000000000038813"},{"key":"2026091506555459401_j_dx-2026-0069_ref_012","doi-asserted-by":"crossref","unstructured":"Jiang, N, Zhang, Y, Liang, S, Lyu, X, Chen, S, Huang, X, et al.. Effectiveness of virtual simulations versus mannequins and real persons in medical and nursing education: meta-analysis and trial sequential analysis of randomized controlled trials. J\u00a0Med Internet Res 2024;26:e56195. https:\/\/doi.org\/10.2196\/56195.","DOI":"10.2196\/56195"},{"key":"2026091506555459401_j_dx-2026-0069_ref_013","doi-asserted-by":"crossref","unstructured":"Cook, DA. Creating virtual patients using large language models: scalable, global, and low cost. Med Teach 2025;47:40\u20132. https:\/\/doi.org\/10.1080\/0142159X.2024.2376879.","DOI":"10.1080\/0142159X.2024.2376879"},{"key":"2026091506555459401_j_dx-2026-0069_ref_014","doi-asserted-by":"crossref","unstructured":"Hicke, Y, Geathers, J, Vu, K, Sewell, J, Cardie, C, Talwalkar, J, et al.. MedSimAI: simulation and formative feedback generation to enhance deliberate practice in medical education. In: Proceedings of the LAK26: 16th International Learning Analytics and Knowledge Conference; 2026: 415\u201324 pp.","DOI":"10.1145\/3785022.3785092"},{"key":"2026091506555459401_j_dx-2026-0069_ref_015","doi-asserted-by":"crossref","unstructured":"Plackett, R, Kassianos, AP, Mylan, S, Kambouri, M, Raine, R, Sheringham, J. The effectiveness of using virtual patient educational tools to improve medical students\u2019 clinical reasoning skills: a systematic review. BMC Med Educ 2022;22:365. https:\/\/doi.org\/10.1186\/s12909-022-03410-x.","DOI":"10.1186\/s12909-022-03410-x"},{"key":"2026091506555459401_j_dx-2026-0069_ref_016","doi-asserted-by":"crossref","unstructured":"Wang, Z, Fan, TT, Li, ML, Zhu, NJ, Wang, XC. Feasibility study of using GPT for history-taking training in medical education: a randomized clinical trial. BMC Med Educ 2025;25:1030. https:\/\/doi.org\/10.1186\/s12909-025-07614-9.","DOI":"10.1186\/s12909-025-07614-9"},{"key":"2026091506555459401_j_dx-2026-0069_ref_017","doi-asserted-by":"crossref","unstructured":"Turner, L, Kelleher, M, Overla, S, Zheng, W, Gregath, A, Gharib, M, et al.. Harnessing the generative power of AI to move closer to personalized medical education. Acad Med 2025. https:\/\/doi.org\/10.1097\/ACM.0000000000006185.","DOI":"10.1097\/ACM.0000000000006185"},{"key":"2026091506555459401_j_dx-2026-0069_ref_018","doi-asserted-by":"crossref","unstructured":"Cook, DA, Overgaard, J, Pankratz, VS, Del Fiol, G, Aakre, CA. Virtual patients using large language models: scalable, contextualized simulation of clinician-patient dialogue with feedback. J\u00a0Med Internet Res 2025;27:e68486. https:\/\/doi.org\/10.2196\/68486.","DOI":"10.2196\/68486"},{"key":"2026091506555459401_j_dx-2026-0069_ref_019","doi-asserted-by":"crossref","unstructured":"Yamamoto, A, Koda, M, Ogawa, H, Miyoshi, T, Maeda, Y, Otsuka, F, et al.. Enhancing medical interview skills through AI-simulated patient interactions: nonrandomized controlled trial. JMIR Med Educ 2024;10:e58753. https:\/\/doi.org\/10.2196\/58753.","DOI":"10.2196\/58753"},{"key":"2026091506555459401_j_dx-2026-0069_ref_020","doi-asserted-by":"crossref","unstructured":"Brugge, E, Ricchizzi, S, Arenbeck, M, Keller, MN, Schur, L, Stummer, W, et al.. Large language models improve clinical decision making of medical students through patient simulation and structured feedback: a randomized controlled trial. BMC Med Educ 2024;24:1391. https:\/\/doi.org\/10.1186\/s12909-024-06399-7.","DOI":"10.1186\/s12909-024-06399-7"},{"key":"2026091506555459401_j_dx-2026-0069_ref_021","doi-asserted-by":"crossref","unstructured":"Liu, Y, Shi, C, Wu, L, Lin, X, Chen, X, Zhu, Y, et al.. Development and validation of a large language model-based system for medical history-taking training: prospective multicase study on evaluation stability, human-AI consistency, and transparency. JMIR Med Educ 2025;11:e73419. https:\/\/doi.org\/10.2196\/73419.","DOI":"10.2196\/73419"},{"key":"2026091506555459401_j_dx-2026-0069_ref_022","doi-asserted-by":"crossref","unstructured":"Li, D, Lebai Lutfi, S. Large language model-based virtual patient systems for history-taking in medical education: comprehensive systematic review. JMIR Med Inform 2026;14:e79039. https:\/\/doi.org\/10.2196\/79039.","DOI":"10.2196\/79039"},{"key":"2026091506555459401_j_dx-2026-0069_ref_023","unstructured":"Zheng, W, Turner, L, Kropczynski, J, Ozer, M, Kelleher, M, Overla, S, et al.. The user-centered iterative design of an LLM-powered educational scenario simulator for clinical reasoning. CS & IT Conf Proc 2025;15."},{"key":"2026091506555459401_j_dx-2026-0069_ref_024","unstructured":"Zheng, W, Turner, L, Kropczynski, J, Ozer, M, Overla, S, Halse, S. A fuzzy supervisor agent design for clinical reasoning assistance in a multi-agent educational clinical scenario simulation. arXiv 2025."},{"key":"2026091506555459401_j_dx-2026-0069_ref_025","doi-asserted-by":"crossref","unstructured":"Holderried, F, Stegemann-Philipps, C, Herrmann-Werner, A, Festl-Wietek, T, Holderried, M, Eickhoff, C, et al.. A language model-powered simulated patient with automated feedback for history taking: prospective study. JMIR Med Educ 2024;10:e59213. https:\/\/doi.org\/10.2196\/59213.","DOI":"10.2196\/59213"},{"key":"2026091506555459401_j_dx-2026-0069_ref_026","doi-asserted-by":"crossref","unstructured":"Holderried, F, Stegemann-Philipps, C, Herschbach, L, Moldt, JA, Nevins, A, Griewatz, J, et al.. A generative pretrained transformer (GPT)-powered chatbot as a simulated patient to practice history taking: prospective, mixed methods study. JMIR Med Educ 2024;10:e53961. https:\/\/doi.org\/10.2196\/53961.","DOI":"10.2196\/53961"},{"key":"2026091506555459401_j_dx-2026-0069_ref_027","doi-asserted-by":"crossref","unstructured":"Zhang, B, Liu, X, Wang, Y, Lei, Z, Xie, Q, Wang, B. Human or LLM as standardized patients? A comparative study in medical education. Assoc Comput Linguist 2026:26988\u20137012.","DOI":"10.18653\/v1\/2026.acl-long.1243"},{"key":"2026091506555459401_j_dx-2026-0069_ref_028","unstructured":"Wattenberg KLaTLaNBaDBaFVaHPaM. Measuring and Controlling Instruction (In)Stability in Language Model Dialogs. presented at: COLM 2024; 2024. https:\/\/arxiv.org\/abs\/2402.10962."},{"key":"2026091506555459401_j_dx-2026-0069_ref_029","doi-asserted-by":"crossref","unstructured":"Cook, DA, Durning, SJ, Sherbino, J, Gruppen, LD. Management reasoning: implications for health professions educators and a research agenda. Acad Med 2019;94:1310\u201316. https:\/\/doi.org\/10.1097\/ACM.0000000000002768.","DOI":"10.1097\/ACM.0000000000002768"}],"container-title":["Diagnosis"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.degruyterbrill.com\/document\/doi\/10.1515\/dx-2026-0069\/xml","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.degruyterbrill.com\/document\/doi\/10.1515\/dx-2026-0069\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,9,15]],"date-time":"2026-09-15T06:56:22Z","timestamp":1789455382000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.degruyterbrill.com\/document\/doi\/10.1515\/dx-2026-0069\/html"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,9,16]]},"references-count":29,"alternative-id":["10.1515\/dx-2026-0069"],"URL":"https:\/\/doi.org\/10.1515\/dx-2026-0069","relation":{},"ISSN":["2194-8011","2194-802X"],"issn-type":[{"value":"2194-8011","type":"print"},{"value":"2194-802X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,9,16]]}}}