{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,30]],"date-time":"2026-09-30T05:04:01Z","timestamp":1790744641638,"version":"4.1.0"},"reference-count":37,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2026,9,30]],"date-time":"2026-09-30T00:00:00Z","timestamp":1790726400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Med."],"abstract":"<jats:sec>\n                    <jats:title>Background<\/jats:title>\n                    <jats:p>Clinical history-taking is a core medical competency, but conventional teaching may be limited by patient availability, variable clinical exposure, and restricted opportunities for repeated practice. Large language models (LLMs) may support simulated practice; however, their clinical, linguistic, and cultural suitability requires careful evaluation in multilingual settings.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Objective<\/jats:title>\n                    <jats:p>This study aimed to develop and expert-evaluate a culturally and linguistically adapted English\u2013Malay clinical dialogue corpus and to descriptively benchmark a fine-tuned LLM framework for clinical history-taking simulation in Malaysian undergraduate medical education.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Methods<\/jats:title>\n                    <jats:p>A multiphase mixed-methods development and evaluation study was conducted across nine internal medicine systems. Of 63,250 AI-generated dialogue entries, each comprising a simulated doctor inquiry and corresponding patient response, 59,184 (93.6%) underwent consultant-level subject matter expert review using an author-developed holistic 5-point rubric informed by clinical relevance, accuracy, clarity and coherence, completeness, and human-likeness. Free-text comments were analysed inductively. After SME evaluation and clinical-quality and mechanical filtering, 58,886 entries across 2,497 simulated patient profiles were retained. The profiles were partitioned into 1,997 training profiles, 250 validation profiles, and 250 held-out test profiles. Four model configurations were descriptively benchmarked using the same held-out profiles and an automated large language model judge. This benchmark was intended for internal model comparison and did not constitute independent external or human clinical validation of the post-fine-tuning model outputs.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>Mean subject matter expert scores were 4.52\/5 for simulated doctor inquiries and 4.58\/5 for simulated patient responses. Qualitative analysis identified recurring limitations involving question structure, incomplete probing, repetition, linguistic accuracy, over-explanation, and simulated-patient realism. In descriptive benchmarking, the fine-tuned model achieved the highest observed mean score among the four configurations evaluated.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusions<\/jats:title>\n                    <jats:p>Developing a consultant-reviewed, culturally adapted English\u2013Malay clinical dialogue corpus and using it to fine-tune an LLM for simulated history-taking was feasible. The fine-tuned model achieved the highest observed score in the internal automated benchmark, but this finding does not establish statistical, clinical, or educational superiority. The post-fine-tuning model outputs were not independently evaluated by clinicians in the present study. The framework may offer a scalable, contextually relevant supplement to clinical communication training, but blinded human evaluation of post-fine-tuning outputs, targeted safety assessment, external validation, and learner-outcome evaluation are required before routine student-facing implementation.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.3389\/fmed.2026.1895321","type":"journal-article","created":{"date-parts":[[2026,9,30]],"date-time":"2026-09-30T04:47:58Z","timestamp":1790743678000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["Development and expert evaluation of a culturally adapted large language model for Malay clinical history-taking simulation: mixed methods study"],"prefix":"10.3389","volume":"13","author":[{"given":"Noor Akmal Shareela","family":"Ismail","sequence":"first","affiliation":[{"name":"Faculty of Medicine, Universiti Kebangsaan Malaysia","place":["Kuala Lumpur, Malaysia"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ahmad Hathim","family":"Ahmad Azman","sequence":"additional","affiliation":[{"name":"Faculty of Medicine, Universiti Kebangsaan Malaysia","place":["Kuala Lumpur, Malaysia"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Azizah","family":"Zafira","sequence":"additional","affiliation":[{"name":"Faculty of Medicine, Universiti Kebangsaan Malaysia","place":["Kuala Lumpur, Malaysia"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Muhammad Faris","family":"Ahmad Faiz","sequence":"additional","affiliation":[{"name":"Centre for Artificial Intelligence Technology (CAIT), Faculty of Information Technology, Universiti Kebangsaan Malaysia","place":["Bangi, Selangor, Malaysia"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Syahrul Sazliyana","family":"Shahahir","sequence":"additional","affiliation":[{"name":"Faculty of Medicine, Universiti Kebangsaan Malaysia","place":["Kuala Lumpur, Malaysia"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ruslinda","family":"Mustafar","sequence":"additional","affiliation":[{"name":"Faculty of Medicine, Universiti Kebangsaan Malaysia","place":["Kuala Lumpur, Malaysia"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Chairozila","family":"Mohd Shamsuddin","sequence":"additional","affiliation":[{"name":"Faculty of Liberal Science, Universiti Kebangsaan Malaysia","place":["Bangi, Selangor, Malaysia"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Daing Zairi","family":"Ma\u2019arof","sequence":"additional","affiliation":[{"name":"Faculty of Liberal Science, Universiti Kebangsaan Malaysia","place":["Bangi, Selangor, Malaysia"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Shahnorbanun","family":"Sahran","sequence":"additional","affiliation":[{"name":"Centre for Artificial Intelligence Technology (CAIT), Faculty of Information Technology, Universiti Kebangsaan Malaysia","place":["Bangi, Selangor, Malaysia"]}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1965","published-online":{"date-parts":[[2026,9,30]]},"reference":[{"key":"B1","doi-asserted-by":"publisher","first-page":"902","DOI":"10.1097\/ACM.0000000000002618","article-title":"Clinical reasoning assessment methods: a scoping review and practical guidance","volume":"94","author":"Daniel","year":"2019","journal-title":"Acad Med"},{"key":"B2","doi-asserted-by":"publisher","first-page":"961","DOI":"10.1111\/medu.12775","article-title":"How to improve the teaching of clinical reasoning: a narrative review","volume":"49","author":"Schmidt","year":"2015","journal-title":"Med Educ"},{"key":"B3","doi-asserted-by":"publisher","first-page":"1057","DOI":"10.1111\/j.1365-2923.2010.03750.x","article-title":"The effectiveness of self-directed learning in health professions education: a systematic review","volume":"44","author":"Murad","year":"2010","journal-title":"Med Educ"},{"key":"B4","doi-asserted-by":"publisher","first-page":"41","DOI":"10.1111\/j.1365-2923.1979.tb00918.x","article-title":"Assessment of clinical competence using an objective structured clinical examination (OSCE)","volume":"13","author":"Harden","year":"1979","journal-title":"Med Educ"},{"key":"B5","doi-asserted-by":"publisher","first-page":"1134","DOI":"10.1111\/j.1365-2923.2003.01717.x","article-title":"OSCE! variations on a theme by Harden","volume":"37","author":"Hodges","year":"2003","journal-title":"Med Educ"},{"key":"B6","doi-asserted-by":"publisher","first-page":"478","DOI":"10.3109\/0142159X.2011.565828","article-title":"Programmatic assessment: from assessment of learning to assessment for learning","volume":"33","author":"Schuwirth","year":"2011","journal-title":"Med Teach"},{"key":"B7","doi-asserted-by":"publisher","first-page":"325","DOI":"10.1111\/medu.12355","article-title":"Competency-based medical education: origins, perspectives and potentialities","volume":"48","author":"ten Cate","year":"2014","journal-title":"Med Educ"},{"key":"B8","doi-asserted-by":"publisher","first-page":"161","DOI":"10.1097\/SIH.0b013e318182fc56","article-title":"Strengths and weaknesses of simulated and real patients in the teaching of skills to medical students: a review","volume":"3","author":"Bokken","year":"2008","journal-title":"Simul Healthc"},{"key":"B9","doi-asserted-by":"publisher","first-page":"e71224","DOI":"10.7759\/cureus.71224","article-title":"The evolution from standardized to virtual patients in medical education","volume":"16","author":"Hamilton","year":"2024","journal-title":"Cureus"},{"key":"B10","doi-asserted-by":"publisher","first-page":"e13930","DOI":"10.2196\/13930","article-title":"Applications and challenges of implementing artificial intelligence in medical education: integrative review","volume":"5","author":"Chan","year":"2019","journal-title":"JMIR Med Educ"},{"key":"B11","doi-asserted-by":"publisher","first-page":"e17158","DOI":"10.2196\/17158","article-title":"Conversational agents in health care: scoping review and conceptual analysis","volume":"22","author":"Tudor Car","year":"2020","journal-title":"J Med Internet Res"},{"key":"B12","doi-asserted-by":"publisher","first-page":"91","DOI":"10.1186\/1472-6920-10-91","article-title":"Virtual patient simulation: what do students make of it? A focus group study","volume":"10","author":"Botezatu","year":"2010","journal-title":"BMC Med Educ"},{"key":"B13","doi-asserted-by":"publisher","first-page":"258","DOI":"10.1038\/s41746-024-01258-7","article-title":"A framework for human evaluation of large language models in healthcare derived from literature review","volume":"7","author":"Tam","year":"2024","journal-title":"NPJ Digit Med"},{"key":"B14","doi-asserted-by":"publisher","first-page":"55","DOI":"10.1016\/j.ctro.2019.04.002","article-title":"Healthcare ex Machina: are conversational agents ready for prime time in oncology?","volume":"16","author":"Bibault","year":"2019","journal-title":"Clin Transl Radiat Oncol"},{"key":"B15","doi-asserted-by":"publisher","first-page":"e0000198","DOI":"10.1371\/journal.pdig.0000198","article-title":"Performance of ChatGPT on USMLE: potential for AI-assisted medical education using large language models","volume":"2","author":"Kung","year":"2023","journal-title":"PLOS Digit Health"},{"key":"B16","doi-asserted-by":"publisher","first-page":"172","DOI":"10.1038\/s41586-023-06291-2","article-title":"Large language models encode clinical knowledge","volume":"620","author":"Singhal","year":"2023","journal-title":"Nature"},{"key":"B17","author":"Nori","year":"2023"},{"key":"B18","doi-asserted-by":"publisher","first-page":"1233","DOI":"10.1056\/NEJMsr2214184","article-title":"Benefits, limits, and risks of GPT-4 as an AI chatbot for medicine","volume":"388","author":"Lee","year":"2023","journal-title":"N Engl J Med"},{"key":"B19","doi-asserted-by":"publisher","first-page":"489","DOI":"10.3390\/info16060489","article-title":"A systematic review of large language models in medical specialties: applications, challenges and future directions","volume":"16","author":"Alkalbani","year":"2025","journal-title":"Information"},{"key":"B20","doi-asserted-by":"publisher","first-page":"26","DOI":"10.1038\/s43856-024-00717-2","article-title":"Current applications and challenges in large language models for patient care: a systematic review","volume":"5","author":"Busch","year":"2025","journal-title":"Commun Med"},{"key":"B21","year":"2025"},{"key":"B22","doi-asserted-by":"publisher","first-page":"299","DOI":"10.3390\/languages7040299","article-title":"Explaining the diversity in Malay-English code-switching patterns: the contribution of typological similarity and bilingual optimization strategies","volume":"7","author":"Treffers-Daller","year":"2022","journal-title":"Languages"},{"key":"B23","doi-asserted-by":"publisher","first-page":"13129","DOI":"10.3390\/app132413129","article-title":"Parallel-based corpus annotation for Malay health documents","volume":"13","author":"Hafsah","year":"2023","journal-title":"Appl Sci"},{"key":"B24","author":"Nazri","year":"2024"},{"key":"B25","author":"Zolkepli","year":"2024"},{"key":"B26","doi-asserted-by":"publisher","first-page":"80","DOI":"10.1186\/s13104-025-07157-8","article-title":"The feasibility of using generative artificial intelligence for history taking in virtual patients","volume":"18","author":"Yi","year":"2025","journal-title":"BMC Res Notes"},{"key":"B27","volume-title":"Year 3 Internal Medicine Study Guide 2024\/2025","year":"2024"},{"key":"B28","volume-title":"Macleod\u2019s Clinical Examination","author":"Dover","year":"2023"},{"key":"B29","doi-asserted-by":"publisher","first-page":"e59213","DOI":"10.2196\/59213","article-title":"A language model-powered simulated patient with automated feedback for history taking: prospective study","volume":"10","author":"Holderried","year":"2024","journal-title":"JMIR Med Educ"},{"key":"B30","doi-asserted-by":"publisher","first-page":"e73419","DOI":"10.2196\/73419","article-title":"Development and validation of a large language model-based system for medical history-taking training: prospective multicase study on evaluation stability, human-AI consistency, and transparency","volume":"11","author":"Liu","year":"2025","journal-title":"JMIR Med Educ"},{"key":"B31","doi-asserted-by":"publisher","first-page":"e68486","DOI":"10.2196\/68486","article-title":"Virtual patients using large language models: scalable, contextualized simulation of clinician-patient dialogue with feedback","volume":"27","author":"Cook","year":"2025","journal-title":"J Med Internet Res"},{"key":"B32","article-title":"LoRA: low-rank adaptation of large language models","author":"Hu","year":"2022"},{"key":"B33","doi-asserted-by":"publisher","first-page":"46595","DOI":"10.52202\/075280-2020","article-title":"Judging LLM-as-a-judge with MT-Bench and Chatbot Arena","volume":"36","author":"Zheng","year":"2023","journal-title":"Adv Neural Inf Process Syst"},{"key":"B34","doi-asserted-by":"publisher","first-page":"101253","DOI":"10.1016\/j.xinn.2025.101253","article-title":"A survey on LLM-as-a-judge","volume":"7","author":"Gu","year":"2026","journal-title":"Innovation"},{"key":"B35","doi-asserted-by":"publisher","first-page":"e81271","DOI":"10.2196\/81271","article-title":"Large language model-based patient simulation to foster communication skills in health care professionals: user-centered development and usability study","volume":"11","author":"Elhilali","year":"2025","journal-title":"JMIR Med Educ"},{"key":"B36","doi-asserted-by":"publisher","first-page":"e79039","DOI":"10.2196\/79039","article-title":"Large language model-based virtual patient systems for history-taking in medical education: comprehensive systematic review","volume":"14","author":"Li","year":"2026","journal-title":"JMIR Med Inform"},{"key":"B37","author":"Voigt","year":"2025"}],"container-title":["Frontiers in Medicine"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fmed.2026.1895321\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,9,30]],"date-time":"2026-09-30T04:48:15Z","timestamp":1790743695000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fmed.2026.1895321\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,9,30]]},"references-count":37,"alternative-id":["10.3389\/fmed.2026.1895321"],"URL":"https:\/\/doi.org\/10.3389\/fmed.2026.1895321","relation":{},"ISSN":["2296-858X"],"issn-type":[{"value":"2296-858X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,9,30]]},"article-number":"1895321"}}