{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,15]],"date-time":"2026-10-15T17:53:47Z","timestamp":1792086827066,"version":"4.4.0"},"reference-count":21,"publisher":"JMIR Publications Inc.","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["JMIR Form Res"],"abstract":"Abstract<\/jats:title>\n \n Background<\/jats:title>\n Multiple-choice examinations are widely used in dental and medical education. To reduce the effects of random guessing, several scoring methods incorporating penalty scores have been proposed. However, it remains unclear whether such scoring methods improve the accuracy of pass or fail decisions when pass marks are appropriately calibrated.<\/jats:p>\n <\/jats:sec>\n \n Objective<\/jats:title>\n \n This study aimed to examine pass or fail classification accuracy under 3 scoring methods (dichotomous scoring, formula scoring, and right-minus-wrong scoring) in summative examinations using single-choice Type A items with 5 answer options and to evaluate the effects of test length and the distribution of the model parameter\n k<\/jats:italic>\n (knowledge-based response probability).\n <\/jats:p>\n <\/jats:sec>\n \n Methods<\/jats:title>\n \n Monte Carlo simulations and analytical derivations were used to evaluate examinations with varying test lengths of up to 300 items. Pass marks were calibrated to the same model-defined cutoff for all scoring methods. Classification accuracy, sensitivity, specificity, and misclassification rates were analyzed across different distributions of\n k<\/jats:italic>\n .\n <\/jats:p>\n <\/jats:sec>\n \n Results<\/jats:title>\n \n With every item answered and pass marks calibrated to the same model-defined cutoff, the 3 scoring methods produced identical pass or fail classifications, as expected from their linear relationship. Classification accuracy increased with test length but was strongly influenced by the distribution of\n k<\/jats:italic>\n . Under a uniform distribution, mean accuracy reached 0.90 after 24 items and 0.95 after 94 items, whereas substantially longer tests were required when\n k<\/jats:italic>\n was concentrated near the cutoff. These item numbers are model- and distribution-dependent and should not be interpreted as recommended examination lengths. Misclassification was highest near the decision threshold.\n <\/jats:p>\n <\/jats:sec>\n \n Conclusions<\/jats:title>\n \n Under the modeled conditions of complete responding, calibrated pass marks, and unchanged response behavior, the investigated scoring methods yielded identical pass or fail classifications. Classification accuracy increased with test length and depended strongly on the distribution of\n k<\/jats:italic>\n , with greatest uncertainty near the pass or fail cutoff. These findings apply to the structural effects of the scoring transformations examined here and should not be generalized to behavioral effects of scoring rules in real examinations.\n <\/jats:p>\n <\/jats:sec>","DOI":"10.2196\/98358","type":"journal-article","created":{"date-parts":[[2026,9,22]],"date-time":"2026-09-22T05:00:06Z","timestamp":1790053206000},"page":"e98358-e98358","source":"Crossref","is-referenced-by-count":0,"title":["Effects of Test Length and Scoring Methods on Pass or Fail Classification Accuracy in Single-Choice Examinations: Simulation Study"],"prefix":"10.2196","volume":"10","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-5950-2716","authenticated-orcid":false,"given":"Christiane","family":"Pink","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0008-0214-6325","authenticated-orcid":false,"given":"Yola","family":"Meisel","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0009-3953-429X","authenticated-orcid":false,"given":"Arvid","family":"Vetter","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2169-561X","authenticated-orcid":false,"given":"Philipp","family":"Kanzow","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1010","published-online":{"date-parts":[[2026,10,15]]},"reference":[{"issue":"2","key":"R1","doi-asserted-by":"publisher","first-page":"63","DOI":"10.1037\/h0073542","article-title":"The Kansas silent reading tests","volume":"7","author":"Kelly","journal-title":"J Educ Psychol"},{"issue":"1","key":"R2","doi-asserted-by":"publisher","first-page":"33","DOI":"10.1080\/00220671.1920.10879021","article-title":"A new kind of school examination","volume":"1","author":"McCall","journal-title":"J Educ Res"},{"key":"R3","doi-asserted-by":"publisher","DOI":"10.2196\/58126","article-title":"Use of multiple-choice items in summative examinations: questionnaire survey among German undergraduate dental training programs","volume":"10","author":"R\u00f6ssler","journal-title":"JMIR Med Educ"},{"key":"R4","doi-asserted-by":"publisher","DOI":"10.2196\/44084","article-title":"Scoring single-response multiple-choice items: scoping review and comparison of different scoring methods","volume":"9","author":"Kanzow","journal-title":"JMIR Med Educ"},{"issue":"8","key":"R5","doi-asserted-by":"publisher","first-page":"670","DOI":"10.1046\/j.1365-2923.2003.01585.x","article-title":"Guessing on selected-response examinations","volume":"37","author":"Downing","journal-title":"Med Educ"},{"issue":"2","key":"R6","doi-asserted-by":"publisher","first-page":"110","DOI":"10.1080\/0142159042000196141","article-title":"Multiple choice questions revisited","volume":"26","author":"Anderson","journal-title":"Med Teach"},{"key":"R7","doi-asserted-by":"publisher","DOI":"10.4137\/JMECD.S40705","article-title":"Negative marking and the student physician--a descriptive study of Nigerian medical schools","volume":"3","author":"Ndu","journal-title":"J Med Educ Curric Dev"},{"issue":"3","key":"R8","doi-asserted-by":"publisher","first-page":"169","DOI":"10.1026\/0033-3042\/a000218","article-title":"Gutachten zur Erstellung \u201egerichtsfester\u201c Multiple-Choice-Pr\u00fcfungsaufgaben [Article in German]","volume":"65","author":"Kubinger","journal-title":"Psychologische Rundschau"},{"issue":"3","key":"R9","first-page":"26","volume":"48","author":"Kirstges","journal-title":"Neue Hochschule"},{"issue":"7","key":"R10","doi-asserted-by":"publisher","first-page":"445","DOI":"10.1037\/h0073083","article-title":"On scoring multiple response tests","volume":"15","author":"Holzinger","journal-title":"J Educ Psychol"},{"issue":"2","key":"R11","doi-asserted-by":"publisher","first-page":"111","DOI":"10.1111\/j.1745-3984.1988.tb00296.x","article-title":"Multiple true\u2010false items: a study of interitem correlations, scoring alternatives, and reliability estimation","volume":"25","author":"Albanese","journal-title":"J Educ Meas"},{"key":"R12","doi-asserted-by":"crossref","unstructured":"Harrell FE . Regression Modeling Strategies: With Applications to Linear Models, Logistic and Ordinal Regression, and Survival Analysis. 2nd ed. Springer; 2015. [doi: 10.1007\/978-3-319-19425-7]","DOI":"10.1007\/978-3-319-19425-7"},{"key":"R13","doi-asserted-by":"publisher","DOI":"10.1016\/j.edurev.2021.100409","article-title":"Relation between examinees\u2019 true knowledge and examination scores: systematic review and exemplary calculations on multiple-true-false items","volume":"34","author":"Schmidt","journal-title":"Educ Res Rev"},{"key":"R14","unstructured":"Embretson SE Reise SP . Item Response Theory for Psychologists. 1st ed. Psychology Press; 2000. ISBN: 9780805828184"},{"key":"R15","unstructured":"Hambleton RK Swaminathan H Rogers HJ . Fundamentals of Item Response Theory. SAGE Publications; 1991. ISBN: 9780803936461"},{"key":"R16","unstructured":"Mood AM Graybill FA Boes DC . Introduction to the Theory of Statistics. 3rd ed. McGraw-Hill; 1974. ISBN: 9780070854659"},{"key":"R17","unstructured":"R Core Team. R: a language and environment for statistical computing. R Foundation for Statistical Computing. URL: https:\/\/www.r-project.org\/ [Accessed 05-01-2026]"},{"key":"R18","unstructured":"DeGroot MH Schervish MJ . Probability and Statistics. 4th ed. Addison-Wesley; 2012. ISBN: 9780321500465"},{"issue":"3","key":"R19","doi-asserted-by":"publisher","first-page":"425","DOI":"10.2307\/1170678","article-title":"Validating the performance standards associated with passing scores","volume":"64","author":"Kane","journal-title":"Rev Educ Res"},{"issue":"2","key":"R20","doi-asserted-by":"publisher","first-page":"179","DOI":"10.1111\/j.1745-3984.1995.tb00462.x","article-title":"Estimating the consistency and accuracy of classifications based on test scores","volume":"32","author":"Livingston","journal-title":"J Educ Meas"},{"issue":"1","key":"R21","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1111\/j.1745-3984.2009.00096.x","article-title":"Classification consistency and accuracy for complex assessments using item response theory","volume":"47","author":"Lee","journal-title":"J Educ Meas"}],"container-title":["JMIR Formative Research"],"original-title":[],"language":"en","deposited":{"date-parts":[[2026,10,15]],"date-time":"2026-10-15T17:41:12Z","timestamp":1792086072000},"score":1,"resource":{"primary":{"URL":"https:\/\/formative.jmir.org\/2026\/1\/e98358"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,10,15]]},"references-count":21,"URL":"https:\/\/doi.org\/10.2196\/98358","relation":{},"ISSN":["2561-326X"],"issn-type":[{"value":"2561-326X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,10,15]]},"article-number":"v10i2e98358"}}