{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,25]],"date-time":"2026-08-25T20:33:37Z","timestamp":1787690017609,"version":"build-2784847793"},"reference-count":8,"publisher":"Oxford University Press (OUP)","issue":"3","license":[{"start":{"date-parts":[[2026,1,25]],"date-time":"2026-01-25T00:00:00Z","timestamp":1769299200000},"content-version":"unspecified","delay-in-days":0,"URL":"https:\/\/academic.oup.com\/pages\/standard-publication-reuse-rights"}],"content-domain":{"domain":["lww.com","ovid.com"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2026,3,1]]},"abstract":"Abstract<\/jats:title>\n \n Problem<\/jats:title>\n High-stakes licensing exams such as the United States Medical Licensing Examination (USMLE) play a critical role in medical education, influencing both trainee progression and patient outcomes. Access to high-quality board preparation resources is uneven and often cost-prohibitive, disproportionately affecting students from underrepresented or financially disadvantaged backgrounds.<\/jats:p>\n <\/jats:sec>\n \n Approach<\/jats:title>\n An artificial intelligence (AI)-driven system to generate USMLE-style practice questions aligned with National Board of Medical Examiners (NBME) item-writing guidelines using a Large Language Model (LLM) enhanced with retrieval augmented generation, chain-of-thought and few-shot prompting, and JavaScript Object Notation schema validation was developed and piloted at the University of Cincinnati College of Medicine between November and December 2023. Five lectures from a preclinical hematology course were selected, and 565 questions were generated for 177 first-year medical students. A human-in-the-loop process, led by a faculty course director, ensured content validity and adherence to educational standards. Validated questions were deployed via a mobile app, allowing students to practice, receive performance feedback, and access an AI tutor.<\/jats:p>\n <\/jats:sec>\n \n Outcomes<\/jats:title>\n Of the 565 questions, 490 (87%) were deemed accurate and NBME-compliant. Eighty students used the question bank, completing up to 220 questions each. Although not statistically significant, increased use trended toward improved performance on related exam questions. Qualitative feedback highlighted enthusiasm for AI-assisted study tools, with calls for broader content coverage.<\/jats:p>\n <\/jats:sec>\n \n Next Steps<\/jats:title>\n This pilot demonstrates that LLMs can generate high-quality, guideline-aligned practice questions. To improve scalability and reduce faculty workload, future iterations will incorporate AI-based review agents for pre-screening content. The platform is intended to be expanded to additional courses, training phases, and health professions. Ongoing refinement will focus on improving content specificity and maintaining accuracy, especially in advanced and subspecialty education.<\/jats:p>\n <\/jats:sec>","DOI":"10.1093\/acamed\/wvaf074","type":"journal-article","created":{"date-parts":[[2026,1,25]],"date-time":"2026-01-25T17:19:50Z","timestamp":1769361590000},"page":"279-283","update-policy":"https:\/\/doi.org\/10.1097\/lww.0000000000001000","source":"Crossref","is-referenced-by-count":4,"title":["An artificial intelligence-driven platform for practice question generation"],"prefix":"10.1093","volume":"101","author":[{"ORCID":"https:\/\/orcid.org\/0009-0009-6456-5258","authenticated-orcid":false,"given":"Andrew","family":"Zahn","sequence":"first","affiliation":[{"name":"University of Cincinnati College of Medicine, Cincinnati, OH, United States"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0001-3696-5395","authenticated-orcid":false,"given":"Seth","family":"Overla","sequence":"additional","affiliation":[{"name":"Office of Artificial Intelligence and Educational Informatics, Department of Biostatistics, Health Informatics, and Data Sciences, University of Cincinnati College of Medicine, Cincinnati, OH, United States"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3842-906X","authenticated-orcid":false,"given":"D J","family":"Lowrie","sequence":"additional","affiliation":[{"name":"Department of Medical Education, University of Cincinnati College of Medicine, OH, United States"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2143-1972","authenticated-orcid":false,"given":"Christine Y","family":"Zhou","sequence":"additional","affiliation":[{"name":"Division of Pulmonary, Critical Care, and Sleep Medicine, University of Cincinnati College of Medicine, Cincinnati, OH, United States"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8327-8002","authenticated-orcid":false,"given":"Sally A","family":"Santen","sequence":"additional","affiliation":[{"name":"Department of Medical Education, University of Cincinnati College of Medicine, OH, United States"},{"name":"Department of Emergency Medicine, University of Cincinnati College of Medicine, Cincinnati, OH, United States"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6496-1906","authenticated-orcid":false,"given":"Weibing","family":"Zheng","sequence":"additional","affiliation":[{"name":"School of Information Technology, University of Cincinnati College of Education, Criminal Justice, and Human Services, Cincinnati, OH, United States"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4567-1313","authenticated-orcid":false,"given":"Laurah","family":"Turner","sequence":"additional","affiliation":[{"name":"Office of Artificial Intelligence and Educational Informatics, Department of Biostatistics, Health Informatics, and Data Sciences, University of Cincinnati College of Medicine, Cincinnati, OH, United States"},{"name":"Department of Medical Education, University of Cincinnati College of Medicine, OH, United States"},{"name":"Department of Biostatistics, Health Informatics, and Data Sciences, University of Cincinnati College of Medicine, Cincinnati, OH, United States"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"286","published-online":{"date-parts":[[2026,3]]},"reference":[{"key":"wvaf074-B1-20260406","doi-asserted-by":"crossref","first-page":"S67","DOI":"10.1097\/ACM.0000000000001916","article-title":"Study behaviors and USMLE step 1 performance: implications of a student self-\u00addirected parallel curriculum","volume":"92","author":"Burk-Rafel","year":"2017","journal-title":"Acad Med"},{"key":"wvaf074-B2-20260406","first-page":"e4168","article-title":"The cost of board examination and preparation: an overlooked factor in medical student debt","volume":"11","author":"Bhatnagar","year":"2019","journal-title":"Cureus"},{"key":"wvaf074-B3-20260406","first-page":"3","article-title":"Racial and socioeconomic inequity in the financial stress of medical school","volume":"12","author":"McMichael","year":"2022","journal-title":"MedEdPublish"},{"key":"wvaf074-B4-20260406","doi-asserted-by":"crossref","first-page":"508","DOI":"10.1097\/ACM.0000000000005626","article-title":"Large language models in medical education: comparing ChatGPT- to human-generated exam questions","volume":"99","author":"Laupichler","year":"2024","journal-title":"Acad Med"},{"key":"wvaf074-B5-20260406","doi-asserted-by":"crossref","first-page":"e0290691","DOI":"10.1371\/journal.pone.0290691","article-title":"ChatGPT versus human in generating medical graduate exam multiple choice questions\u2014a multinational prospective study (Hong Kong S.A.R., Singapore, Ireland, and the United Kingdom)","volume":"18","author":"Cheung","year":"2023","journal-title":"PLoS One"},{"key":"wvaf074-B6-20260406","doi-asserted-by":"crossref","first-page":"1018","DOI":"10.1080\/0142159X.2024.2314723","article-title":"Case-based MCQ generator: a custom ChatGPT based on published prompts in the literature for automatic item generation","volume":"46","author":"K\u0131yak","year":"2024","journal-title":"Med Teach"},{"key":"wvaf074-B7-20260406","doi-asserted-by":"crossref","first-page":"1","DOI":"10.3109\/0142159X.2011.595437","article-title":"Building bridges between theory and practice in medical education using a design-based \u00adresearch approach: AMEE guide No. 60","volume":"34","author":"Dolmans","year":"2012","journal-title":"Med Teach"},{"key":"wvaf074-B10-20260406","doi-asserted-by":"crossref","first-page":"e288","DOI":"10.3109\/0142159X.2012.668637","article-title":"Program evaluation models and related theories: AMEE guide No. 67","volume":"34","author":"Frye","year":"2012","journal-title":"Med Teach"}],"container-title":["Academic Medicine"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/academicmedicine\/advance-article-pdf\/doi\/10.1093\/acamed\/wvaf074\/66569470\/wvaf074.pdf","content-type":"application\/pdf","content-version":"am","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/academicmedicine\/article-pdf\/101\/3\/279\/66569470\/wvaf074.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/journals.lww.com\/10.1093\/acamed\/wvaf074","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,4,14]],"date-time":"2026-04-14T09:45:51Z","timestamp":1776159951000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/academicmedicine\/article\/101\/3\/279\/8440749"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,1,25]]},"references-count":8,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2026]]}},"URL":"https:\/\/doi.org\/10.1093\/acamed\/wvaf074","relation":{},"ISSN":["1938-808X"],"issn-type":[{"value":"1938-808X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,1,25]]},"assertion":[{"value":"2025-10-24","name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}}]}}