{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,23]],"date-time":"2026-10-23T20:38:04Z","timestamp":1792787884351,"version":"4.5.0"},"reference-count":35,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2026,9,24]],"date-time":"2026-09-24T00:00:00Z","timestamp":1790208000000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2026,9,24]],"date-time":"2026-09-24T00:00:00Z","timestamp":1790208000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"name":"Universit\u00e4t des Saarlandes","id":[{"id":"https:\/\/ror.org\/01jdpyv68","id-type":"ROR","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["BMC Med Educ"],"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:sec>\n                    <jats:title>Background<\/jats:title>\n                    <jats:p>Beyond technical feasibility of Artificial Intelligence (AI)-generated multiple-choice questions (MCQs), their educational value in assessment remains unclear. This study aims to evaluate whether students can distinguish between AI-MCQs and National Licensing Exam (NLE) questions in an exam setting and if they align with the curriculum.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Methods<\/jats:title>\n                    <jats:p>In this cross-sectional study 119 year five medical students completed a Family Medicine MCQ tablet-based exam. Participants answered 30 AI-generated and 30 NLE MCQs. AI questions were generated from digital learning materials using ChatGPT-4o and Gemini 1.5 Pro. Experts accepted 82% of the questions, with minor edits to retained items and removing those needing major changes. During the exam, students were asked about the question source and whether each item aligned with the course curriculum. Statistical analyses were obtained using Jamovi 2.3.28.0.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>\n                      No significant difference in correct attribution of AI or NLE MCQs (t(29.6)\u2009=\u2009-1.24,\n                      <jats:italic>p<\/jats:italic>\n                      \u2009=\u2009.225; t(29.6)\u2009=\u20091.18,\n                      <jats:italic>p<\/jats:italic>\n                      \u2009=\u2009.246) was observed. No significant correlation was found between item difficulty and recognition (\u03c4_b:\n                      <jats:italic>p<\/jats:italic>\n                      \u2009=\u2009.534).\u00a0Distractor distributions did not differ across ChatGPT, Google Gemini and NLE (\u03c7\n                      <jats:sup>2<\/jats:sup>\n                      (2)\u2009=\u20092.61,\n                      <jats:italic>p<\/jats:italic>\n                      \u2009=\u2009.271, U\n                      <jats:italic>p<\/jats:italic>\n                      \u2009&gt;\u2009.05 for all comparisons). Item difficulty did not differ significantly between AI and NLE items; however, exploratory source-specific analysis showed an overall difference among ChatGPT, Google Gemini, and NLE items (\u03c7\u00b2(2) = 6.71,\n                      <jats:italic>p<\/jats:italic>\n                      = .035), with a difference between Google Gemini and NLE items (\n                      <jats:italic>p<\/jats:italic>\n                      = .028). Student-perceived curricular alignment did not differ significantly between AI and NLE or among ChatGPT, Google Gemini, and NLE.\n                    <\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusions<\/jats:title>\n                    <jats:p>Students\u2019 recognition did not differ between AI-generated MCQs and NLE MCQs. Easier MCQs are generally perceived as more aligned with the curriculum. AI-MCQs may provide a feasible approach to item drafting within a structured human-review process.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.1186\/s12909-026-10410-8","type":"journal-article","created":{"date-parts":[[2026,9,24]],"date-time":"2026-09-24T06:04:30Z","timestamp":1790229870000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Can students identify AI? \u2013 A cross-sectional quantitative study about AI recognition in tablet-based MCQ assessment among fifth-year undergraduate medical students at Saarland University, Germany"],"prefix":"10.1186","volume":"26","author":[{"given":"Philip","family":"Vogt","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Nadine","family":"Wolf","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sandra","family":"Jordan","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sara","family":"Volz-Willems","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Johannes","family":"J\u00e4ger","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Fabian","family":"Dupont","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2026,9,24]]},"reference":[{"key":"10410_CR1","doi-asserted-by":"publisher","first-page":"433","DOI":"10.1093\/MIND\/LIX.236.433","volume":"LIX","author":"AM Turing","year":"1950","unstructured":"Turing AM. I.\u2014computing machinery and intelligence. Mind. 1950;LIX:433\u201360. https:\/\/doi.org\/10.1093\/MIND\/LIX.236.433.","journal-title":"Mind"},{"key":"10410_CR2","doi-asserted-by":"publisher","unstructured":"Nov O, Singh N, Mann D. Putting ChatGPT\u2019s Medical Advice to the (Turing) Test: Survey Study. JMIR Med Educ. 2023;9. https:\/\/doi.org\/10.2196\/46939.","DOI":"10.2196\/46939"},{"issue":"1","key":"10410_CR3","doi-asserted-by":"publisher","first-page":"354","DOI":"10.1186\/S12909-024-05239-Y","volume":"24","author":"Y Artsi","year":"2024","unstructured":"Artsi Y, Sorin V, Konen E, Glicksberg BS, Nadkarni G, Klang E. Large language models for generating medical examinations: systematic review. BMC Med Educ. 2024;24(1):354. https:\/\/doi.org\/10.1186\/S12909-024-05239-Y.","journal-title":"BMC Med Educ"},{"key":"10410_CR4","doi-asserted-by":"publisher","unstructured":"Law AKK, So J, Lui CT, Choi YF, Cheung KH, Kei-ching Hung K, et al. AI versus human-generated multiple-choice questions for medical education: a cohort study in a high-stakes examination. BMC Med Educ. 2025;25. https:\/\/doi.org\/10.1186\/S12909-025-06796-6.","DOI":"10.1186\/S12909-025-06796-6"},{"key":"10410_CR5","doi-asserted-by":"publisher","first-page":"347","DOI":"10.1007\/BF00138871\/METRICS","volume":"32","author":"J Biggs","year":"1996","unstructured":"Biggs J. Enhancing teaching through constructive alignment. High Educ (Dordr). 1996;32:347\u201364. https:\/\/doi.org\/10.1007\/BF00138871\/METRICS.","journal-title":"High Educ (Dordr)"},{"key":"10410_CR6","doi-asserted-by":"publisher","first-page":"166.e7","DOI":"10.1016\/J.AMJMED.2005.10.036","volume":"119","author":"DA Cook","year":"2006","unstructured":"Cook DA, Beckman TJ. Current concepts in validity and reliability for psychometric instruments: theory and application. Am J Med. 2006;119:166.e7-166.e16. https:\/\/doi.org\/10.1016\/J.AMJMED.2005.10.036.","journal-title":"Am J Med"},{"key":"10410_CR7","doi-asserted-by":"publisher","unstructured":"Royal KD, Hedgpeth M-W, Jeon T, Colford CM. Automated Item Generation: The Future of Medical Education Assessment? EMJ Innov. 2018;2:88\u201393. https:\/\/doi.org\/10.33590\/EMJINNOV\/10313113.","DOI":"10.33590\/EMJINNOV\/10313113"},{"key":"10410_CR8","doi-asserted-by":"publisher","unstructured":"Magzoub ME, Zafar I, Munshi F, Shersad F. Ten tips to harnessing generative AI for high-quality MCQS in medical education assessment. Med Educ Online. 2025;30. https:\/\/doi.org\/10.1080\/10872981.2025.2532682;PAGE:STRING:ARTICLE\/CHAPTER.","DOI":"10.1080\/10872981.2025.2532682;PAGE:STRING:ARTICLE\/CHAPTER"},{"key":"10410_CR9","doi-asserted-by":"publisher","first-page":"1224","DOI":"10.1080\/0142159X.2023.2249239","volume":"45","author":"M Zuckerman","year":"2023","unstructured":"Zuckerman M, Flood R, Tan RJB, Kelp N, Ecker DJ, Menke J, et al. ChatGPT for assessment writing. Med Teach. 2023;45:1224\u20137. https:\/\/doi.org\/10.1080\/0142159X.2023.2249239.","journal-title":"Med Teach"},{"key":"10410_CR10","doi-asserted-by":"publisher","unstructured":"Reid M, French M, Andreopoulos S, Wong C, Kee N. AI-generated multiple-choice questions in health science education: Stakeholder perspectives and implementation considerations. Curr Res Physiol. 2025;8. https:\/\/doi.org\/10.1016\/J.CRPHYS.2025.100160.","DOI":"10.1016\/J.CRPHYS.2025.100160"},{"key":"10410_CR11","doi-asserted-by":"publisher","first-page":"103","DOI":"10.3946\/KJME.2023.253","volume":"35","author":"S Ahn","year":"2023","unstructured":"Ahn S. The impending impacts of large language models on medical education. Korean J Med Educ. 2023;35:103. https:\/\/doi.org\/10.3946\/KJME.2023.253.","journal-title":"Korean J Med Educ"},{"key":"10410_CR12","doi-asserted-by":"publisher","unstructured":"Turner L, Knopp MI, Mendonca EA, Desai S. Bridging Artificial Intelligence and Medical Education Navigating the Alignment Paradox. ATS Sch. 2025;6:135\u201348. https:\/\/doi.org\/10.34197\/ATS-SCHOLAR.2024-0086PS\/SUPPL_FILE\/DISCLOSURES.PDF.","DOI":"10.34197\/ATS-SCHOLAR.2024-0086PS\/SUPPL_FILE\/DISCLOSURES.PDF"},{"key":"10410_CR13","doi-asserted-by":"publisher","first-page":"101491","DOI":"10.1016\/J.TSC.2024.101491","volume":"52","author":"M Squalli Houssaini","year":"2024","unstructured":"Squalli Houssaini M, Aboutajeddine A, Toughrai I, Ibrahimi A. Development of a design course for medical curriculum: Using design thinking as an instructional design method empowered by constructive alignment and generative AI. Think Skills Creat. 2024;52:101491. https:\/\/doi.org\/10.1016\/J.TSC.2024.101491.","journal-title":"Think Skills Creat"},{"key":"10410_CR14","doi-asserted-by":"publisher","first-page":"352","DOI":"10.1007\/S40037-018-0480-3","volume":"7","author":"M Ricci","year":"2018","unstructured":"Ricci M, St-Onge C, Xiao J, Young M. Students as stakeholders in assessment: how students perceive the value of an assessment. Perspect Med Educ. 2018;7:352\u201361. https:\/\/doi.org\/10.1007\/S40037-018-0480-3.","journal-title":"Perspect Med Educ"},{"key":"10410_CR15","doi-asserted-by":"publisher","first-page":"875","DOI":"10.3109\/0142159X.2011.595434","volume":"33","author":"J Sandars","year":"2011","unstructured":"Sandars J, Cleary TJ. Self-regulation theory: applications to medical education: AMEE guide no. 58. Med Teach. 2011;33:875\u201386. https:\/\/doi.org\/10.3109\/0142159X.2011.595434.","journal-title":"Med Teach"},{"key":"10410_CR16","doi-asserted-by":"publisher","unstructured":"Thesen T, Park SH. A generative AI teaching assistant for personalized learning in medical education. npj Digital Medicine 2025;8(1):627-. https:\/\/doi.org\/10.1038\/s41746-025-02022-1.","DOI":"10.1038\/s41746-025-02022-1"},{"key":"10410_CR17","doi-asserted-by":"publisher","first-page":"275","DOI":"10.1007\/S10459-006-9039-X","volume":"13","author":"J Sargeant","year":"2008","unstructured":"Sargeant J, Mann K, Sinclair D, Van Der Vleuten C, Metsemakers J. Understanding the influence of emotions and reflection upon multi-source feedback acceptance and use. Adv Health Sci Educ Theory Pract. 2008;13:275\u201388. https:\/\/doi.org\/10.1007\/S10459-006-9039-X.","journal-title":"Adv Health Sci Educ Theory Pract"},{"key":"10410_CR18","doi-asserted-by":"publisher","unstructured":"Kaya M, Sonmez E, Halici A, Yildirim H, Coskun A. Comparison of AI-generated and clinician-designed multiple-choice questions in emergency medicine exam: a psychometric analysis. BMC Med Educ. 2025;25:949-. https:\/\/doi.org\/10.1186\/S12909-025-07528-6\/FIGURES\/2.","DOI":"10.1186\/S12909-025-07528-6\/FIGURES\/2"},{"key":"10410_CR19","doi-asserted-by":"publisher","first-page":"1082","DOI":"10.3102\/0034654317726529","volume":"87","author":"MJ Gierl","year":"2017","unstructured":"Gierl MJ, Bulut O, Guo Q, Zhang X. Developing, analyzing, and using distractors for multiple-choice tests in education: a comprehensive review. Rev Educ Res. 2017;87:1082\u2013116. https:\/\/doi.org\/10.3102\/0034654317726529.","journal-title":"Rev Educ Res"},{"key":"10410_CR20","doi-asserted-by":"publisher","first-page":"508","DOI":"10.1097\/ACM.0000000000005626","volume":"99","author":"MC Laupichler","year":"2024","unstructured":"Laupichler MC, Rother JF, Grunwald Kadow IC, Ahmadi S, Raupach T. Large language models in medical education: comparing ChatGPT- to human-generated exam questions. Acad Med. 2024;99:508\u201312. https:\/\/doi.org\/10.1097\/ACM.0000000000005626.","journal-title":"Acad Med"},{"key":"10410_CR21","doi-asserted-by":"publisher","first-page":"1276","DOI":"10.1111\/MEDU.15402","volume":"58","author":"HC Lucas","year":"2024","unstructured":"Lucas HC, Upperman JS, Robinson JR. A systematic review of large language models and their implications in medical education. Med Educ. 2024;58:1276\u201385. https:\/\/doi.org\/10.1111\/MEDU.15402.","journal-title":"Med Educ"},{"key":"10410_CR22","doi-asserted-by":"publisher","first-page":"e0000198","DOI":"10.1371\/journal.pdig.0000198","volume":"2","author":"TH Kung","year":"2023","unstructured":"Kung TH, Cheatham M, Medenilla A, Sillos C, De Leon L, Elepa\u00f1o C, et al. Performance of ChatGPT on USMLE: potential for AI-assisted medical education using large language models. PLoS Digit Health. 2023;2:e0000198. https:\/\/doi.org\/10.1371\/journal.pdig.0000198.","journal-title":"PLoS Digit Health"},{"key":"10410_CR23","doi-asserted-by":"publisher","unstructured":"Deutscher \u00c4rzteverlag \u00a9, Zfa | |. Can a Family Medicine Curriculum Increase the Attraction of Family Medicine as a Career Choice? Kann ein Allgemeinmedizincurriculum f\u00fcr eine Weiterbildung im Fach Allgemeinmedizin motivieren? Zeitschrift f\u00fcr Allgemeinmedizin |. 2022;98. https:\/\/doi.org\/10.53180\/zfa.2022.0229-0233.","DOI":"10.53180\/zfa.2022.0229-0233"},{"key":"10410_CR24","unstructured":"Praktische Hinweise - www.impp.de. Institut f\u00fcr medizinische und pharmazeutische Pr\u00fcfungsfragen. 2025. https:\/\/www.impp.de\/pruefungen\/allgemein\/praktische-hinweise.html. Accessed 12 Aug 2026."},{"key":"10410_CR25","doi-asserted-by":"publisher","first-page":"S15","DOI":"10.1097\/ACM.0000000000006107","volume":"100","author":"CK Boscardin","year":"2025","unstructured":"Boscardin CK, Abdulnour REE, Gin BC. Macy Foundation Innovation Report Part I: current landscape of artificial intelligence in medical education. Acad Med. 2025;100:S15-21. https:\/\/doi.org\/10.1097\/ACM.0000000000006107.","journal-title":"Acad Med"},{"key":"10410_CR26","doi-asserted-by":"publisher","first-page":"S22","DOI":"10.1097\/ACM.0000000000006117","volume":"100","author":"BC Gin","year":"2025","unstructured":"Gin BC, Laforge K, Burk-Rafel J, Boscardin CK. Macy Foundation Innovation Report Part II: from hype to reality: innovators\u2019 visions for navigating AI integration challenges in medical education. Acad Med. 2025;100:S22\u20139. https:\/\/doi.org\/10.1097\/ACM.0000000000006117.","journal-title":"Acad Med"},{"key":"10410_CR27","doi-asserted-by":"publisher","unstructured":"Jackson FI, Keller NA, Kouba I, Kouba W, Bracero LA, Blitz MJ. Large Language Model Clinical Vignettes and Multiple-Choice Questions for Postgraduate Medical Education. Academic Medicine. 2025. https:\/\/doi.org\/10.1097\/ACM.0000000000006137.","DOI":"10.1097\/ACM.0000000000006137"},{"key":"10410_CR28","doi-asserted-by":"publisher","first-page":"116","DOI":"10.1097\/ACM.0000000000005760","volume":"100","author":"M Lie","year":"2025","unstructured":"Lie M, Rodman A, Crowe B. Harnessing generative artificial intelligence for medical education. Acad Med. 2025;100:116. https:\/\/doi.org\/10.1097\/ACM.0000000000005760.","journal-title":"Acad Med"},{"key":"10410_CR29","doi-asserted-by":"publisher","first-page":"149","DOI":"10.1055\/s-0045-1813040","volume":"15","author":"A Mufti","year":"2025","unstructured":"Mufti A, Ali K, Sharma G, Saleem M. Human expertise outperforms artificial intelligence in medical education assessments: MCQ creation highlights the irreplaceable role of teachers. Avicenna J Med. 2025;15:149\u201356. https:\/\/doi.org\/10.1055\/s-0045-1813040.","journal-title":"Avicenna J Med"},{"issue":"1","key":"10410_CR30","doi-asserted-by":"publisher","DOI":"10.1038\/s41746-025-01786-w","volume":"8","author":"T Templin","year":"2025","unstructured":"Templin T, Fort S, Padmanabham P, Seshadri P, Rimal R, Oliva J, et al. Framework for bias evaluation in large language models in healthcare settings. npj Digit Med. 2025;8(1):414-. https:\/\/doi.org\/10.1038\/s41746-025-01786-w.","journal-title":"npj Digit Med"},{"key":"10410_CR31","doi-asserted-by":"publisher","unstructured":"Rodman A, Mark NM, Artino AR, Lessing JN. Using generative artificial intelligence in medical education. Academic Medicine. 2024. https:\/\/doi.org\/10.1097\/ACM.0000000000005937.","DOI":"10.1097\/ACM.0000000000005937"},{"key":"10410_CR32","doi-asserted-by":"publisher","unstructured":"Chenot JF. Undergraduate medical education in Germany. Ger Med Sci. 2009;7. https:\/\/doi.org\/10.3205\/000061.","DOI":"10.3205\/000061"},{"key":"10410_CR33","doi-asserted-by":"publisher","first-page":"32","DOI":"10.1016\/J.HPE.2016.08.001","volume":"3","author":"EK Chang","year":"2017","unstructured":"Chang EK, Wimmers PF. Effect of repeated\/spaced formative assessments on medical school final exam performance. Health Prof Educ. 2017;3:32\u20137. https:\/\/doi.org\/10.1016\/J.HPE.2016.08.001.","journal-title":"Health Prof Educ"},{"key":"10410_CR34","doi-asserted-by":"publisher","unstructured":"Tate H, Bicknell BT, Fiore P, Galloway J, Borak J, Brooks WS. Medical students\u2019 perceptions of AI-generated practice questions as learning tools. Journal of Investigative Medicine. 2025. https:\/\/doi.org\/10.1177\/10815589251406265.","DOI":"10.1177\/10815589251406265"},{"key":"10410_CR35","doi-asserted-by":"publisher","unstructured":"Lipnevich AA, Mattern K, Feddock C. Formative assessment and feedback in medical education: A practical guide: AMEE Guide No. 189. Med Teach. 2025. https:\/\/doi.org\/10.1080\/0142159X.2025.2569623.","DOI":"10.1080\/0142159X.2025.2569623"}],"container-title":["BMC Medical Education"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/article\/10.1186\/s12909-026-10410-8","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s12909-026-10410-8.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s12909-026-10410-8.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,9,24]],"date-time":"2026-09-24T08:13:53Z","timestamp":1790237633000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1186\/s12909-026-10410-8"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,9,24]]},"references-count":35,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2026,12]]}},"alternative-id":["10410"],"URL":"https:\/\/doi.org\/10.1186\/s12909-026-10410-8","relation":{},"ISSN":["1472-6920"],"issn-type":[{"value":"1472-6920","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,9,24]]},"assertion":[{"value":"28 May 2026","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"11 September 2026","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"24 September 2026","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"Not applicable.","order":1,"name":"Ethics","label":"Consent for publication","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"FD is current Wonca World Director and executive committee member. He also serves as elected World Health Organisation (WHO) youth council member. He also holds active collaboration agreements with the digtial editing house (Amboss) and the German public national state exam licensing committee (IMPP).","order":2,"name":"Ethics","label":"Competing interests","group":{"name":"EthicsHeading","label":"Declarations"}}],"article-number":"1519"}}