{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,29]],"date-time":"2026-08-29T05:18:48Z","timestamp":1787980728069,"version":"build-2784847793"},"reference-count":32,"publisher":"MDPI AG","issue":"9","license":[{"start":{"date-parts":[[2026,8,27]],"date-time":"2026-08-27T00:00:00Z","timestamp":1787788800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/100000050","name":"NHLBI","doi-asserted-by":"crossref","award":["1K99HL169940"],"award-info":[{"award-number":["1K99HL169940"]}],"id":[{"id":"10.13039\/100000050","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Nurs. Rep.","Nursing Reports"],"abstract":"<jats:p>Background\/Objectives: High-quality multiple-choice questions (MCQs) are essential for valid assessment in graduate-level nursing programs, yet developing such items is resource-intensive for educators. Large language models such as ChatGPT may generate educational assessment content; however, evidence regarding their effectiveness in producing graduate-level nursing examination items remains limited. This study compared the perceived quality of ChatGPT-generated and educator-authored MCQs for graduate-level nursing examinations and examined exploratory associations between assessor characteristics and quality ratings. Methods: A comparative cross-sectional study was conducted among 27 international nurse educators who independently evaluated 50 MCQs (25 ChatGPT-generated and 25 educator-authored) using an expert-reviewed rubric assessing eight quality domains, with the primary outcome being the between-source difference in perceived quality ratings. Participants were blinded to question source. Results: ChatGPT-generated MCQs received nominally higher ratings than educator-authored items across most domains, but effect sizes were uniformly small (Cohen\u2019s d = 0.11\u20130.33). No significant differences were observed in perceived difficulty or distractor plausibility; after Bonferroni correction for multiple domain comparisons, differences in cognitive level, scenario relevance, and instructional alignment remained significant. Conclusions: ChatGPT-generated MCQs achieved perceived-quality ratings broadly comparable to educator-authored items when supported by structured prompting and expert refinement, although expert review remains necessary before classroom use, and these results should be regarded as preliminary. These findings support AI-assisted item development as a complementary strategy pending replication in larger, more rigorous studies.<\/jats:p>","DOI":"10.3390\/nursrep16090304","type":"journal-article","created":{"date-parts":[[2026,8,27]],"date-time":"2026-08-27T10:58:21Z","timestamp":1787828301000},"page":"304","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Can AI Write MCQs? A Comparison of ChatGPT-Generated and Educator-Authored Multiple-Choice Questions in Graduate Nursing Education"],"prefix":"10.3390","volume":"16","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-4865-1190","authenticated-orcid":false,"given":"Nader","family":"Alnomasy","sequence":"first","affiliation":[{"name":"Medical Surgical Nursing Department, College of Nursing, University of Hail, Hail 81451, Saudi Arabia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9121-268X","authenticated-orcid":false,"given":"Habib","family":"Alrashedi","sequence":"additional","affiliation":[{"name":"Medical Surgical Nursing Department, College of Nursing, University of Hail, Hail 81451, Saudi Arabia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8766-1527","authenticated-orcid":false,"given":"Maha","family":"Dardouri","sequence":"additional","affiliation":[{"name":"Community Health Nursing Department, College of Nursing, University of Hail, Hail 81451, Saudi Arabia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Soha Kamel Mosbah","family":"Mahmoud","sequence":"additional","affiliation":[{"name":"Community Health Nursing Department, College of Nursing, University of Hail, Hail 81451, Saudi Arabia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Petelyne","family":"Pangket","sequence":"additional","affiliation":[{"name":"Department of Community and Mental Health Nursing Sciences, Taif University, Taif 21944, Saudi Arabia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1424-4140","authenticated-orcid":false,"given":"Sang Bin","family":"You","sequence":"additional","affiliation":[{"name":"Department of Biobehavioral Health Sciences, University of Pennsylvania School of Nursing, Philadelphia, PA 19104, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7671-8532","authenticated-orcid":false,"given":"Aluem","family":"Tark","sequence":"additional","affiliation":[{"name":"Department of Nursing, Helene Fuld College of Nursing, New York, NY 10035, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Deborah","family":"Becker","sequence":"additional","affiliation":[{"name":"Department of Biobehavioral Health Sciences, University of Pennsylvania School of Nursing, Philadelphia, PA 19104, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0280-2248","authenticated-orcid":false,"given":"Richard Balacuit","family":"Maestrado","sequence":"additional","affiliation":[{"name":"Medical Surgical Nursing Department, College of Nursing, University of Hail, Hail 81451, Saudi Arabia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"suffix":"Jr.","given":"Romeo","family":"Mostoles","sequence":"additional","affiliation":[{"name":"Department of Psychiatric and Mental Health Nursing, College of Nursing, University of Hail, Hail 81451, Saudi Arabia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4967-9803","authenticated-orcid":false,"given":"Rayhanah R.","family":"Almutairi","sequence":"additional","affiliation":[{"name":"Community and Psychiatric Mental Health Nursing Department, College of Nursing, Princess Nourah Bint Abdulrahman University, Riyadh 11671, Saudi Arabia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0492-7615","authenticated-orcid":false,"given":"Ebtsam","family":"Abouhashish","sequence":"additional","affiliation":[{"name":"College of Nursing, King Saud Bin Abdulaziz University for Health Sciences, Jeddah 21423, Saudi Arabia"},{"name":"King Abdullah International Medical Research Center, Jeddah 22384, Saudi Arabia"},{"name":"Ministry of National Guard Health Affairs, Jeddah 22384, Saudi Arabia"},{"name":"Faculty of Nursing, Alexandria University, Alexandria 21526, Egypt"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7259-769X","authenticated-orcid":false,"given":"Sudharani B.","family":"Banappagoudar","sequence":"additional","affiliation":[{"name":"Department of Nursing, College of Applied Medical Sciences, King Faisal University, Al Ahsa 31982, Saudi Arabia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Asim A.","family":"Alreshidi","sequence":"additional","affiliation":[{"name":"Emergency Department, King Khalid Hospital, Hail Health Cluster, Hail 55421, Saudi Arabia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0000-8081-3922","authenticated-orcid":false,"given":"Waleed Meajib","family":"Alshammari","sequence":"additional","affiliation":[{"name":"Advanced Oncology Nursing, King Faisal Specialist Hospital & Research Center, Riyadh 11211, Saudi Arabia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0005-9134-6545","authenticated-orcid":false,"given":"Faisal Fahaad","family":"Alrshidi","sequence":"additional","affiliation":[{"name":"Department College of Medicine, Sechenova University, Moscow 119048, Russia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Razan","family":"Alsayed","sequence":"additional","affiliation":[{"name":"Nursing Education Department, King Fahd Armed Forces Hospital, Jeddah 23311, Saudi Arabia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sharifah","family":"Alsayed","sequence":"additional","affiliation":[{"name":"College of Nursing, King Saud Bin Abdulaziz University for Health Sciences, Jeddah 21423, Saudi Arabia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0362-0670","authenticated-orcid":false,"given":"Jiyoun","family":"Song","sequence":"additional","affiliation":[{"name":"Department of Biobehavioral Health Sciences, University of Pennsylvania School of Nursing, Philadelphia, PA 19104, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2026,8,27]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Hyun, A., Tower, M., and Turner, C. (2022). The current contexts of newly graduated nurses\u2019 competence: A content analysis. Healthcare, 10.","DOI":"10.3390\/healthcare10061071"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"309","DOI":"10.1207\/S15324818AME1503_5","article-title":"A review of multiple-choice item-writing guidelines for classroom assessment","volume":"15","author":"Haladyna","year":"2002","journal-title":"Appl. Meas. Educ."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"152","DOI":"10.3163\/1536-5050.103.3.010","article-title":"Bloom\u2019s taxonomy of cognitive learning objectives","volume":"103","author":"Adams","year":"2015","journal-title":"J. Med. Libr. Assoc."},{"key":"ref_4","first-page":"e99299","article-title":"Psychometric properties of multiple-choice questions used in undergraduate medical education in India: A systematic review and meta-analysis","volume":"17","author":"Mondal","year":"2025","journal-title":"Cureus"},{"key":"ref_5","first-page":"32","article-title":"Higher order thinking skills assessment (HOTS)","volume":"3","author":"Widana","year":"2017","journal-title":"JISAE J. Indones. Stud. Assess. Eval."},{"key":"ref_6","unstructured":"Benner, P., Hughes, R.G., and Sutphen, M. (2026, July 14). Clinical reasoning, decision making, and action: Thinking critically and clinically, Patient Safety and Quality: An Evidence-Based Handbook for Nurses, Available online: https:\/\/www.ncbi.nlm.nih.gov\/books\/NBK2643\/."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"103548","DOI":"10.1016\/j.nepr.2023.103548","article-title":"Teaching strategies of clinical reasoning in advanced nursing clinical practice: A scoping review","volume":"67","author":"Giuffrida","year":"2023","journal-title":"Nurse Educ. Pract."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"126531","DOI":"10.1016\/j.eswa.2025.126531","article-title":"An assessment framework of higher-order thinking skills based on fine-tuned large language models","volume":"272","author":"Xiao","year":"2025","journal-title":"Expert Syst. Appl."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"210","DOI":"10.1016\/j.teln.2025.01.019","article-title":"Artificial intelligence in nursing education: A review of AI-based teaching pedagogies","volume":"20","author":"Labrague","year":"2025","journal-title":"Teach. Learn. Nurs."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"em2307","DOI":"10.29333\/ejmste\/13428","article-title":"Exploring the potential of artificial intelligence tools in educational measurement and assessment","volume":"19","author":"Owan","year":"2023","journal-title":"Eurasia J. Math. Sci. Technol. Educ."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"1961","DOI":"10.1080\/0142159X.2025.2505122","article-title":"GPT-4 versus human authors in clinically complex MCQ creation: A blinded analysis of item quality","volume":"47","author":"Wu","year":"2025","journal-title":"Med. Teach."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"446","DOI":"10.1080\/0142159X.2024.2314198","article-title":"A scoping review of artificial intelligence in medical education: BEME Guide No. 84","volume":"46","author":"Gordon","year":"2024","journal-title":"Med. Teach."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Law, A.K.K., So, J., Lui, C.T., Choi, Y.F., Cheung, K.H., Kei-Ching Hung, K.K.C., and Graham, C.A. (2025). AI versus human-generated multiple-choice questions for medical education: A cohort study in a high-stakes examination. BMC Med. Educ., 25.","DOI":"10.1186\/s12909-025-06796-6"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Riehm, L., Nanji, K., Lakhani, M., Pankiv, E., Hasanee, D., and Pfeifer, W. (2026). The use of large language models in generating multiple choice questions for health professions education: A systematic review and network meta-analysis. PLoS ONE, 21.","DOI":"10.1371\/journal.pone.0340277"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Almaghaslah, D. (2025). Challenging the curve: Can ChatGPT-generated MCQs reduce grade inflation in pharmacy education. Front. Pharmacol., 16.","DOI":"10.3389\/fphar.2025.1516381"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Cheung, B.H.H., Lau, G.K.K., Wong, G.T.C., Lee, E.Y.P., Kulkarni, D., Seow, C.S., Wong, R., and Co, M.T.H. (2023). ChatGPT versus human in generating medical graduate exam multiple choice questions\u2014A multinational prospective study. PLoS ONE, 18.","DOI":"10.1101\/2023.05.13.23289943"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"261","DOI":"10.1080\/0142159X.2025.2533401","article-title":"Comparing AI-generated and traditional textbook multiple-choice questions in nursing education: A prompt engineering-based Delphi study","volume":"48","author":"Lin","year":"2026","journal-title":"Med. Teach."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Douthit, N.T., Norcini, J., Mazuz, K., Alkan, M., Feuerstein, M.T., Clarfield, A.M., Dwolatzky, T., Solomonov, E., Waksman, I., and Biswas, S. (2021). Assessment of global health education: The role of multiple-choice questions. Front. Public Health, 9.","DOI":"10.3389\/fpubh.2021.640204"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"112","DOI":"10.31729\/jnma.2976","article-title":"Construction of multiple choice questions before and after an educational intervention","volume":"56","author":"Tenzin","year":"2017","journal-title":"JNMA J. Nepal Med. Assoc."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"200","DOI":"10.1080\/23752696.2020.1816843","article-title":"Using rubrics to improve the assessment lifecycle: A case study","volume":"5","author":"Miknis","year":"2020","journal-title":"High. Educ. Pedagog."},{"key":"ref_21","first-page":"3","article-title":"Does an assessment rubric provide a better learning experience for undergraduates in developing transferable skills?","volume":"19","author":"Pang","year":"2022","journal-title":"J. Univ. Teach. Learn. Pract."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"3473","DOI":"10.1111\/jan.16628","article-title":"Exploring the current applications and effectiveness of ChatGPT in nursing: An integrative review","volume":"81","author":"Luo","year":"2025","journal-title":"J. Adv. Nurs."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Hadzhikoleva, S., Rachovski, T., Ivanov, I., Hadzhikolev, E., and Dimitrov, G. (2024). Automated test creation using large language models: A practical application. Appl. Sci., 14.","DOI":"10.3390\/app14199125"},{"key":"ref_24","unstructured":"Chapelle, C., Beckett, G., and Ranalli, J. (2024). A comparative analysis of multiple-choice questions: ChatGPT-generated items vs. human-developed items. Exploring AI in Applied Linguistics, Iowa State University Digital Press."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"679","DOI":"10.3928\/01484834-20231006-07","article-title":"Comparative analysis of NCLEX-RN questions: A duel between ChatGPT and human expertise","volume":"62","author":"Cox","year":"2023","journal-title":"J. Nurs. Educ."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Kaya, M., Sonmez, E., Halici, A., Yildirim, H., and Coskun, A. (2025). Comparison of AI-generated and clinician-designed multiple-choice questions in emergency medicine exam: A psychometric analysis. BMC Med. Educ., 25.","DOI":"10.1186\/s12909-025-07528-6"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"467","DOI":"10.14423\/SMJ.0000000000001715","article-title":"Comparison of the usability and reliability of answers to clinical questions: AI-generated ChatGPT versus a human-authored resource","volume":"117","author":"Manian","year":"2024","journal-title":"South. Med. J."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"19","DOI":"10.1186\/s40468-024-00291-3","article-title":"A comparative study of AI-human-made and human-made test forms for a university TESOL theory course","volume":"14","author":"O","year":"2024","journal-title":"Lang. Test. Asia"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"576","DOI":"10.33225\/pec\/20.78.576","article-title":"Quality and feature of multiple-choice questions in education","volume":"78","author":"Jia","year":"2020","journal-title":"Probl. Educ. 21st Century"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"169","DOI":"10.36959\/447\/356","article-title":"Relationships between age, subjective memory, metacognition and thought-action fusion","volume":"6","author":"Metehan","year":"2022","journal-title":"Ann. Cogn. Sci."},{"key":"ref_31","first-page":"e59778","article-title":"The utility of multiple-choice assessment in current medical education: A critical review","volume":"16","author":"Parekh","year":"2024","journal-title":"Cureus"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Przymusza\u0142a, P., Piotrowska, K., Lipski, D., Marciniak, R., and Cerbin-Koczorowska, M. (2020). Guidelines on writing multiple choice questions: A well-received and effective faculty development intervention. Sage Open, 10.","DOI":"10.1177\/2158244020947432"}],"container-title":["Nursing Reports"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2039-4403\/16\/9\/304\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,8,29]],"date-time":"2026-08-29T04:45:45Z","timestamp":1787978745000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2039-4403\/16\/9\/304"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,8,27]]},"references-count":32,"journal-issue":{"issue":"9","published-online":{"date-parts":[[2026,9]]}},"alternative-id":["nursrep16090304"],"URL":"https:\/\/doi.org\/10.3390\/nursrep16090304","relation":{},"ISSN":["2039-4403"],"issn-type":[{"value":"2039-4403","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,8,27]]}}}