{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,8,21]],"date-time":"2025-08-21T15:40:07Z","timestamp":1755790807322,"version":"3.44.0"},"reference-count":31,"publisher":"American Medical Association (AMA)","issue":"8","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["JAMA Ophthalmol"],"published-print":{"date-parts":[[2025,8,1]]},"abstract":"<jats:sec id=\"ab-eoi250032-4\"><jats:title>Importance<\/jats:title><jats:p>The Ophthalmic Knowledge Assessment Program (OKAP) has been widely used to determine readiness to take the high-stakes Written Qualifying Examination (WQE). However, it is unclear how well OKAP performance can predict WQE outcomes.<\/jats:p><\/jats:sec><jats:sec id=\"ab-eoi250032-5\"><jats:title>Objective<\/jats:title><jats:p>To examine the association between the OKAP and WQE nationally.<\/jats:p><\/jats:sec><jats:sec id=\"ab-eoi250032-6\"><jats:title>Design, Setting, and Participants<\/jats:title><jats:p>This was a retrospective cohort study conducted from 2017 to 2022. This national data analysis included residents from ophthalmology residency programs in the US. Study data were analyzed June 2023 to December 2024.<\/jats:p><\/jats:sec><jats:sec id=\"ab-eoi250032-7\"><jats:title>Exposures<\/jats:title><jats:p>OKAP and WQE data.<\/jats:p><\/jats:sec><jats:sec id=\"ab-eoi250032-8\"><jats:title>Main Outcomes and Measures<\/jats:title><jats:p>The national OKAP examination data and the WQE data were matched. Candidates who had complete data for the OKAP during postgraduate year (PGY) 2, 3, and 4 and completed the WQE were included in the study. Two-sample <jats:italic>t<\/jats:italic> tests, logistic regression analysis, receiver operating characteristic curves, and confusion matrix were used to examine the association of various test scores (PGY-2 OKAP, PGY-3 OKAP, PGY-4 OKAP, minimum OKAP, maximum OKAP, average OKAP) as predictors for WQE outcomes (pass or fail). Multiple logistic regression analyses were also used to assess the association of gender, time since graduation, program size, and WQE pass\/fail outcomes.<\/jats:p><\/jats:sec><jats:sec id=\"ab-eoi250032-9\"><jats:title>Results<\/jats:title><jats:p>A total of 1597 residents (907 male [56.8%]) from 117 residency programs were included in this study. First-time pass rate of the WQE was 89% (1418 of 1597 residents). OKAP scores, particularly average OKAP, maximum OKAP, and PGY-4 OKAP scores (all had area under the curve\u2009=\u20090.88), were the most useful predictor of WQE outcomes. PGY-4 OKAP scores were used for the rest of the analysis for simplicity. A PGY-4 OKAP scaled score of 550 corresponded to 94% of candidates passing the WQE. Using PGY-4 OKAP scores to predict passing\/failing, the WQE had 90.7% accuracy. Gender (odds ratio [OR],\u20091.50; 95% CI, 1.01-2.26; <jats:italic>P<\/jats:italic> =\u2009.048), program size (OR,\u20091.55; 95% CI, 0.98-2.41; <jats:italic>P<\/jats:italic> =\u2009.06), and time since residency graduation (OR,\u20091.00; 95% CI, 1.00-1.00; <jats:italic>P<\/jats:italic> =\u2009.14) were not included in the logistic regression model investigating the association of PGY-4 OKAP scores with the likelihood of passing the WQE.<\/jats:p><\/jats:sec><jats:sec id=\"ab-eoi250032-10\"><jats:title>Conclusions and Relevance<\/jats:title><jats:p>Results suggest that OKAP examination scores were associated with passing the WQE after adjusting for program size, gender, and time since residency graduation. Further analyses may be warranted to predict failing the WQE so that potential interventions can be attempted.<\/jats:p><\/jats:sec>","DOI":"10.1001\/jamaophthalmol.2025.1933","type":"journal-article","created":{"date-parts":[[2025,7,10]],"date-time":"2025-07-10T15:00:34Z","timestamp":1752159634000},"page":"678","source":"Crossref","is-referenced-by-count":0,"title":["Ophthalmic Knowledge Assessment Program Scores and Written Qualifying Examination Performance"],"prefix":"10.1001","volume":"143","author":[{"given":"Chanon","family":"Thanitcul","sequence":"first","affiliation":[{"name":"Department of Ophthalmology and Visual Sciences, University of Michigan Kellogg Eye Center, Ann Arbor"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Sarah","family":"Schnabel","sequence":"additional","affiliation":[{"name":"Department of Educational Psychology, University of Illinois Chicago, Chicago"},{"name":"American Board of Ophthalmology, Doylestown, Pennsylvania"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"George B.","family":"Bartley","sequence":"additional","affiliation":[{"name":"American Board of Ophthalmology, Doylestown, Pennsylvania"},{"name":"Mayo Clinic in Minnesota, Rochester, Minnesota"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Andrew G.","family":"Lee","sequence":"additional","affiliation":[{"name":"Department of Ophthalmology, Blanton Eye Institute, Houston Methodist Hospital, Houston, Texas"},{"name":"Department of Ophthalmology, Weill Cornell Medicine, New York, New York"},{"name":"Department of Ophthalmology, University of Texas Medical Branch, Galveston"},{"name":"University of Texas MD Anderson Cancer Center, Houston"},{"name":"Texas A&M College of Medicine, Bryan"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Hally","family":"Crump","sequence":"additional","affiliation":[{"name":"Data Recognition Corporation, Maple Grove, Minnesota"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Dale E.","family":"Fajardo","sequence":"additional","affiliation":[{"name":"American Academy of Ophthalmology, San Francisco, California"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Kathryn","family":"Peters","sequence":"additional","affiliation":[{"name":"American Academy of Ophthalmology, San Francisco, California"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Beth","family":"Wilson","sequence":"additional","affiliation":[{"name":"American Academy of Ophthalmology, San Francisco, California"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Divya","family":"Srikumaran","sequence":"additional","affiliation":[{"name":"Wilmer Eye Institute, Johns Hopkins University School of Medicine, Baltimore, Maryland"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"10","reference":[{"issue":"4","key":"eoi250032r1","doi-asserted-by":"publisher","first-page":"282","DOI":"10.1016\/0039-6257(88)90176-2","article-title":"The Ophthalmic Knowledge Assessment Program (OKAP): a personal view.","volume":"32","author":"Rubin","year":"1988","journal-title":"Surv Ophthalmol"},{"issue":"1","key":"eoi250032r3","doi-asserted-by":"publisher","first-page":"194","DOI":"10.1016\/S0161-6420(94)31365-0","article-title":"New directions for the Ophthalmic Knowledge Assessment Program (OKAP) examination.","volume":"101","author":"Liesegang","year":"1994","journal-title":"Ophthalmology"},{"issue":"1","key":"eoi250032r4","doi-asserted-by":"publisher","first-page":"190","DOI":"10.1186\/s12909-019-1637-4","article-title":"Resident and program characteristics that impact performance on the Ophthalmic Knowledge Assessment Program (OKAP).","volume":"19","author":"Zafar","year":"2019","journal-title":"BMC Med Educ"},{"issue":"1","key":"eoi250032r5","doi-asserted-by":"publisher","first-page":"43","DOI":"10.4300\/JGME-D-20-00111.1","article-title":"A systematic review of the relationship between in-training examination scores and specialty board examination scores.","volume":"13","author":"McCrary","year":"2021","journal-title":"J Grad Med Educ"},{"issue":"6","key":"eoi250032r6","doi-asserted-by":"publisher","first-page":"641","DOI":"10.1016\/j.amepre.2011.08.008","article-title":"The in-service examination score as a predictor of success on the American Board of Preventive Medicine certification examination.","volume":"41","author":"Bedno","year":"2011","journal-title":"Am J Prev Med"},{"issue":"3","key":"eoi250032r7","doi-asserted-by":"publisher","first-page":"647","DOI":"10.1007\/s13187-016-0993-6","article-title":"Medical knowledge assessment by hematology and medical oncology in-training examinations are better than program director assessments at predicting subspecialty certification examination performance.","volume":"32","author":"Collichio","year":"2017","journal-title":"J Cancer Educ"},{"issue":"23","key":"eoi250032r8","doi-asserted-by":"publisher","first-page":"2862","DOI":"10.1016\/j.jacc.2017.04.020","article-title":"Performance on the cardiovascular in-training examination in relation to the ABIM cardiovascular disease certification examination.","volume":"69","author":"Indik","year":"2017","journal-title":"J Am Coll Cardiol"},{"issue":"1","key":"eoi250032r9","doi-asserted-by":"publisher","first-page":"100","DOI":"10.1097\/ACM.0000000000000500","article-title":"The relationship between internal medicine residency graduate performance on the ABIM certifying examination, yearly in-service training examinations, and the USMLE Step 1 examination.","volume":"90","author":"Kay","year":"2015","journal-title":"Acad Med"},{"issue":"12","key":"eoi250032r11","doi-asserted-by":"publisher","first-page":"2435","DOI":"10.1016\/j.ophtha.2010.03.056","article-title":"A comparative study of resident performance on standardized training examinations and the American Board of Ophthalmology written examination.","volume":"117","author":"Johnson","year":"2010","journal-title":"Ophthalmology"},{"issue":"10","key":"eoi250032r12","doi-asserted-by":"publisher","first-page":"1949","DOI":"10.1016\/j.ophtha.2012.06.010","article-title":"A multicenter analysis of the ophthalmic knowledge assessment program and American Board of Ophthalmology written qualifying examination performance.","volume":"119","author":"Lee","year":"2012","journal-title":"Ophthalmology"},{"issue":"9","key":"eoi250032r13","doi-asserted-by":"publisher","first-page":"2194","DOI":"10.1007\/s11606-022-07597-z","article-title":"Association of gender and race\/ethnicity with internal medicine in-training examination performance in graduate medical education.","volume":"37","author":"Klein","year":"2022","journal-title":"J Gen Intern Med"},{"issue":"5","key":"eoi250032r14","doi-asserted-by":"publisher","first-page":"639","DOI":"10.1016\/j.surg.2011.12.004","article-title":"The state of performance on the American Board of Surgery Qualifying Examination and Certifying Examination and the effect of residency program size on program pass rates.","volume":"151","author":"Falcone","year":"2012","journal-title":"Surgery"},{"issue":"5","key":"eoi250032r15","doi-asserted-by":"publisher","first-page":"483","DOI":"10.1177\/0009922812436550","article-title":"Compliance on the American Board of Pediatrics certifying examination and the importance of location and size on pass rates.","volume":"51","author":"Falcone","year":"2012","journal-title":"Clin Pediatr (Phila)"},{"issue":"1","key":"eoi250032r16","doi-asserted-by":"publisher","first-page":"22","DOI":"10.1001\/jamasurg.2019.4081","article-title":"Association of demographic and program factors with American Board of Surgery qualifying and certifying examinations pass rates.","volume":"155","author":"Yeo","year":"2020","journal-title":"JAMA Surg"},{"issue":"1","key":"eoi250032r19","doi-asserted-by":"publisher","DOI":"10.1080\/10872981.2022.2152162","article-title":"Resident factors associated with American Board of Internal Medicine certification exam failure.","volume":"28","author":"Seaberg","year":"2023","journal-title":"Med Educ Online"},{"issue":"6","key":"eoi250032r20","doi-asserted-by":"publisher","first-page":"e144","DOI":"10.1016\/j.jsurg.2014.04.004","article-title":"Using the American Board of Surgery in-training examination to predict board certification: a cautionary study.","volume":"71","author":"Jones","year":"2014","journal-title":"J Surg Educ"},{"issue":"5","key":"eoi250032r21","doi-asserted-by":"publisher","first-page":"677","DOI":"10.1093\/cid\/ciu906","article-title":"The relationship between performance on the infectious diseases in-training and certification examinations.","volume":"60","author":"Grabovsky","year":"2015","journal-title":"Clin Infect Dis"},{"issue":"5","key":"eoi250032r22","doi-asserted-by":"publisher","first-page":"710","DOI":"10.2215\/CJN.05580517","article-title":"Performance on the nephrology in-training examination and ABIM Nephrology Certification Examination outcomes.","volume":"13","author":"Jurich","year":"2018","journal-title":"Clin J Am Soc Nephrol"},{"issue":"1","key":"eoi250032r23","doi-asserted-by":"publisher","first-page":"212","DOI":"10.1097\/ALN.0b013e3181c62e2f","article-title":"Predicting success on the certification examinations of the American Board of Anesthesiology.","volume":"112","author":"McClintock","year":"2010","journal-title":"Anesthesiology"},{"issue":"6","key":"eoi250032r24","doi-asserted-by":"publisher","first-page":"606","DOI":"10.1177\/000313481908500624","article-title":"A structured remediation program results in durable improvement of American Board of Surgery in-training examination (ABSITE) performance.","volume":"85","author":"Wasicek","year":"2019","journal-title":"Am Surg"},{"issue":"4","key":"eoi250032r25","doi-asserted-by":"publisher","first-page":"754","DOI":"10.1016\/j.jsurg.2017.01.004","article-title":"Orthopedic In-training examination: a performance review based on program- and resident-specific characteristics.","volume":"74","author":"Camp","year":"2017","journal-title":"J Surg Educ"},{"issue":"6","key":"eoi250032r26","doi-asserted-by":"publisher","first-page":"688","DOI":"10.1111\/acem.12391","article-title":"Delaying the American Board of Emergency Medicine qualifying examination is associated with poorer performance.","volume":"21","author":"Marco","year":"2014","journal-title":"Acad Emerg Med"},{"issue":"1","key":"eoi250032r29","doi-asserted-by":"publisher","first-page":"10","DOI":"10.1186\/s12909-018-1440-7","article-title":"Effects of gender in resident evaluations and certifying examination pass rates.","volume":"19","author":"Sulistio","year":"2019","journal-title":"BMC Med Educ"},{"key":"eoi250032r30","doi-asserted-by":"publisher","first-page":"131","DOI":"10.1016\/j.jss.2018.06.014","article-title":"Is there gender bias on the American Board of Surgery General Surgery Certifying Examination?","volume":"237","author":"Ong","year":"2019","journal-title":"J Surg Res"},{"issue":"1","key":"eoi250032r31","doi-asserted-by":"publisher","first-page":"61","DOI":"10.1016\/j.jss.2013.02.040","article-title":"The importance of residency program size and location on American Osteopathic Board of Surgery In-Training Examination outcomes.","volume":"184","author":"Falcone","year":"2013","journal-title":"J Surg Res"},{"key":"eoi250032r2","unstructured":"American Academy of Ophthalmology. OKAP Exam. Accessed October 1, 2023. https:\/\/www.aao.org\/education\/okap-exam"},{"key":"eoi250032r10","unstructured":"Carey? A, Drucker? M. Standardized training examinations among ophthalmology residents and the American Board of Ophthalmology Written Qualifying Examination first attempt: the Morsani College of Medicine experience. Accessed July 14, 2024. https:\/\/digitalcommons.usf.edu\/cgi\/viewcontent.cgi?article=1000&context=oph_facpub"},{"key":"eoi250032r17","unstructured":"American Academy of Ophthalmology. OKAP exam day guide for residents. Accessed July 14, 2024. https:\/\/www.aao.org\/Assets\/7385909b-dbb5-4c3e-86cc-83ae8726ae22\/638723806862970000\/okap-exam-day-guide-for-residents-pdf"},{"key":"eoi250032r18","unstructured":"American Academy of Ophthalmology. Your WQE score report. Accessed May 8, 2024. https:\/\/abop.org\/become-certified\/written-qualifying-exam\/your-wqe-score-report"},{"key":"eoi250032r27","unstructured":"American Academy of Orthopaedic Surgeons. Orthopaedic In-Training Examination (OITE) technical report 2021. Accessed January 1, 2024. https:\/\/www.aaos.org\/globalassets\/education\/product-pages\/oite\/oite-2021-technical-report.pdf"},{"key":"eoi250032r28","unstructured":"American Board of Orthopaedic Surgery. Examination statistics. Accessed January 1, 2024. https:\/\/www.abos.org\/certification\/part-i\/examination-statistics\/"}],"container-title":["JAMA Ophthalmology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/jamanetwork.com\/journals\/jamaophthalmology\/articlepdf\/2835951\/jamaophthalmology_thanitcul_2025_oi_250032_1754685054.6045.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,8,21]],"date-time":"2025-08-21T15:00:43Z","timestamp":1755788443000},"score":1,"resource":{"primary":{"URL":"https:\/\/jamanetwork.com\/journals\/jamaophthalmology\/fullarticle\/2835951"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,8,1]]},"references-count":31,"journal-issue":{"issue":"8","published-print":{"date-parts":[[2025,8,1]]}},"URL":"https:\/\/doi.org\/10.1001\/jamaophthalmol.2025.1933","relation":{},"ISSN":["2168-6165"],"issn-type":[{"type":"print","value":"2168-6165"}],"subject":[],"published":{"date-parts":[[2025,8,1]]}}}