{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,15]],"date-time":"2026-05-15T04:10:21Z","timestamp":1778818221527,"version":"3.51.4"},"reference-count":39,"publisher":"Association of University Professors of Ophthalmology","issue":"02","license":[{"start":{"date-parts":[[2021,11,10]],"date-time":"2021-11-10T00:00:00Z","timestamp":1636502400000},"content-version":"vor","delay-in-days":132,"URL":"https:\/\/creativecommons.org\/licenses\/by-nc-nd\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Journal of Academic Ophthalmology"],"published-print":{"date-parts":[[2021,7]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>\n          Purpose\u2003Only from reviewing applications, it is difficult to identify which applicants will be successful ophthalmology residents. The change of the USMLE Step 1 scoring to \u201cPass\/Fail\u201d removes another quantitative metric. We aimed to identify application attributes correlated with successful residency performance. This study also used artificial intelligence (AI) to evaluate letters of recommendation (LOR), the Dean's letter (MSPE), and personal statement (PS).<\/jats:p><jats:p>\n          Design\u2003Retrospective analysis of application characteristics versus residency performance was conducted.<\/jats:p><jats:p>\n          Participants\u2003Residents who graduated from the Dean McGee Eye Institute\/University of Oklahoma Ophthalmology residency from 2004 to 2019 were included in this study.<\/jats:p><jats:p>\n          Methods\u2003Thirty-four attributes were recorded from each application. Residents were subjectively ranked into tertiles and top and bottom deciles based on residency performance by faculty present during their training. The Ophthalmic Knowledge Assessment Program (OKAP) examination scores were used as an objective performance metric. Analysis was performed to identify associations between application attributes and tertile\/decile ranking. Additional analysis used AI and natural language processing to evaluate applicant LORs, MSPE, and PS.<\/jats:p><jats:p>\n          Main Outcome Measures\u2003Characteristics from residency applications that correlate with resident performance were the primary outcome of this study.<\/jats:p><jats:p>\n          Results\u2003Fifty-five residents and 21 faculty members were included. A grade of \u201cA\u201d or \u201cHonors\u201d in the obstetrics\/gynecology (OB\/GYN) clerkship and the presence of a home ophthalmology department were associated with ranking in the top tertile but not the top decile. Mean core clerkship grades, medical school ranking in the top 25 U.S. News and World Report (USNWR) primary care rankings, and postgraduate year (PGY)-2 and PGY-3 OKAP scores were predictive of being ranked in both the top tertile and the top decile. USMLE scores, alpha-omega-alpha (AOA) status, and number of publications did not correlate with subjective resident performance. AI analysis of LORs, MSPE, and PS did not identify any text features that correlated with resident performance.<\/jats:p><jats:p>\n          Conclusions\u2003Many metrics traditionally felt to be predictive of residency success (USMLE scores, AOA status, and research) did not predict resident success in our study. We did confirm the importance of core clerkship grades and medical school ranking. Objective measures of success such as PGY-2 and PGY-3 OKAP scores were associated with high subjective ranking.<\/jats:p>","DOI":"10.1055\/s-0041-1733932","type":"journal-article","created":{"date-parts":[[2021,11,10]],"date-time":"2021-11-10T18:30:30Z","timestamp":1636569030000},"page":"e151-e157","source":"Crossref","is-referenced-by-count":6,"title":["Correlation of Ophthalmology Residency Application Characteristics with Subsequent Performance in Residency"],"prefix":"10.62199","volume":"13","author":[{"given":"Brett M.","family":"Gudgel","sequence":"additional","affiliation":[{"name":"University of Oklahoma Health Science Center, Oklahoma City, Oklahoma"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Andrew T.","family":"Melson","sequence":"additional","affiliation":[{"name":"University of Oklahoma Health Science Center, Oklahoma City, Oklahoma"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Justin","family":"Dvorak","sequence":"additional","affiliation":[{"name":"University of Oklahoma Health Science Center, Oklahoma City, Oklahoma"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Kai","family":"Ding","sequence":"additional","affiliation":[{"name":"University of Oklahoma Health Science Center, Oklahoma City, Oklahoma"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"R. Michael","family":"Siatkowski","sequence":"additional","affiliation":[{"name":"University of Oklahoma Health Science Center, Oklahoma City, Oklahoma"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"48610","published-online":{"date-parts":[[2021,11,10]]},"reference":[{"issue":"05","key":"ref3","doi-asserted-by":"crossref","first-page":"1041","DOI":"10.1016\/j.ophtha.2009.07.034","article-title":"Ophthalmology resident selection: current trends in selection criteria and improving the process","volume":"117","author":"S Nallasamy","year":"2010","journal-title":"Ophthalmology"},{"issue":"01","key":"ref4","doi-asserted-by":"crossref","first-page":"e150","DOI":"10.1055\/s-0038-1673675","article-title":"Probability of success in the ophthalmology residency match: three-year outcomes analysis of San Francisco matching program data","volume":"10","author":"R M Siatkowski","year":"2018","journal-title":"J Acad Ophthalmol"},{"key":"ref6","doi-asserted-by":"crossref","first-page":"22","DOI":"10.1016\/j.urology.2017.06.046","article-title":"Predictors of a successful urology resident using medical student application materials","volume":"108","author":"R H Thompson","year":"2017","journal-title":"Urology"},{"issue":"06","key":"ref7","doi-asserted-by":"crossref","first-page":"444","DOI":"10.1016\/j.jsurg.2010.05.002","article-title":"The electronic residency application service application can predict accreditation council for graduate medical education competency-based surgical resident performance","volume":"67","author":"A M Tolan","year":"2010","journal-title":"J Surg Educ"},{"issue":"06","key":"ref8","doi-asserted-by":"crossref","first-page":"378","DOI":"10.1016\/j.jsurg.2007.05.003","article-title":"Importance of the faculty interview during the resident application process","volume":"64","author":"T E Brothers","year":"2007","journal-title":"J Surg Educ"},{"issue":"04","key":"ref9","doi-asserted-by":"crossref","first-page":"908","DOI":"10.1007\/s11999-015-4317-7","article-title":"Does residency selection criteria predict performance in orthopaedic surgery residency?","volume":"474","author":"T Raman","year":"2016","journal-title":"Clin Orthop Relat Res"},{"issue":"04","key":"ref10","doi-asserted-by":"crossref","first-page":"437","DOI":"10.1016\/S1076-6332(03)80189-7","article-title":"Does medical school performance predict radiology resident performance?","volume":"9","author":"T D Boyse","year":"2002","journal-title":"Acad Radiol"},{"issue":"02","key":"ref11","first-page":"270","article-title":"Resident selection: applicant selection criteria compared with performance","volume":"92","author":"E E Erlandson","year":"1982","journal-title":"Surgery"},{"issue":"09","key":"ref12","first-page":"834","article-title":"Do the criteria of resident selection committees predict residents' performances?","volume":"70","author":"P L Fine","year":"1995","journal-title":"Acad Med"},{"issue":"S 01","key":"ref13","first-page":"S302","article-title":"Resident recruitment: what are good predictors for performance during pediatric residency training?","volume":"85","author":"A Khongphatthanayothin","year":"2002","journal-title":"J Med Assoc Thai"},{"issue":"399","key":"ref14","doi-asserted-by":"crossref","first-page":"265","DOI":"10.1097\/00003086-200206000-00034","article-title":"Correlating selection criteria with subsequent performance as residents","author":"D R Dirschl","year":"2002","journal-title":"Clin Orthop Relat Res"},{"issue":"02","key":"ref15","doi-asserted-by":"crossref","first-page":"72","DOI":"10.5435\/00124635-201102000-00002","article-title":"Success in orthopaedic training: resident selection and predictors of quality performance","volume":"19","author":"K A Egol","year":"2011","journal-title":"J Am Acad Orthop Surg"},{"issue":"05","key":"ref16","doi-asserted-by":"crossref","first-page":"1010","DOI":"10.1016\/j.jaad.2010.07.034","article-title":"Can performance in medical school predict performance in residency? A compilation and review of correlative studies","volume":"65","author":"K L Harfmann","year":"2011","journal-title":"J Am Acad Dermatol"},{"issue":"11","key":"ref17","doi-asserted-by":"crossref","first-page":"2750","DOI":"10.2106\/JBJS.H.01243","article-title":"Factors associated with successful performance in an orthopaedic surgery residency","volume":"91","author":"A B Spitzer","year":"2009","journal-title":"J Bone Joint Surg Am"},{"issue":"08","key":"ref18","doi-asserted-by":"crossref","first-page":"707","DOI":"10.1001\/archoto.2012.1374","article-title":"Predictors of future success in otolaryngology residency applicants","volume":"138","author":"R A Chole","year":"2012","journal-title":"Arch Otolaryngol Head Neck Surg"},{"issue":"01","key":"ref19","doi-asserted-by":"crossref","first-page":"138","DOI":"10.1016\/j.jsurg.2012.06.015","article-title":"Predictors of success in a urology residency program","volume":"70","author":"S G Grewal","year":"2013","journal-title":"J Surg Educ"},{"issue":"03","key":"ref20","doi-asserted-by":"crossref","first-page":"206","DOI":"10.1197\/j.aem.2005.01.002","article-title":"What characteristics of applicants to emergency medicine residency programs predict future success as an emergency medicine resident?","volume":"12","author":"S R Hayden","year":"2005","journal-title":"Acad Emerg Med"},{"key":"ref21","doi-asserted-by":"crossref","first-page":"69","DOI":"10.1016\/j.clineuro.2015.05.007","article-title":"Predicting success: What medical student measures predict resident performance in neurology?","volume":"135","author":"M J Burish","year":"2015","journal-title":"Clin Neurol Neurosurg"},{"issue":"02","key":"ref22","doi-asserted-by":"crossref","first-page":"502","DOI":"10.1213\/01.ANE.0000154583.47236.5F","article-title":"The resident application process and its correlation to future performance as a resident","volume":"100","author":"D G Metro","year":"2005","journal-title":"Anesth Analg"},{"key":"ref23","doi-asserted-by":"crossref","first-page":"139","DOI":"10.1016\/j.jclinane.2017.03.052","article-title":"The predictive value of pre-recruitment achievement on resident performance in anesthesiology","volume":"39","author":"F Chen","year":"2017","journal-title":"J Clin Anesth"},{"issue":"03","key":"ref24","doi-asserted-by":"crossref","first-page":"322","DOI":"10.4300\/JGME-D-09-00101.1","article-title":"Can medical school performance predict residency performance? Resident selection and predictors of successful performance in obstetrics and gynecology","volume":"2","author":"H E Stohl","year":"2010","journal-title":"J Grad Med Educ"},{"issue":"06","key":"ref25","doi-asserted-by":"crossref","first-page":"513","DOI":"10.1016\/j.jsurg.2011.07.007","article-title":"The predictive value of general surgery application data for future resident performance","volume":"68","author":"D M Alterman","year":"2011","journal-title":"J Surg Educ"},{"issue":"06","key":"ref26","doi-asserted-by":"crossref","first-page":"1011","DOI":"10.1177\/0194599817698430","article-title":"Associations between otolaryngology applicant characteristics and future performance in residency or practice: a systematic review","volume":"156","author":"S N Bowe","year":"2017","journal-title":"Otolaryngol Head Neck Surg"},{"issue":"04","key":"ref27","doi-asserted-by":"crossref","first-page":"649","DOI":"10.1016\/j.jamcollsurg.2005.12.006","article-title":"Predictors for resident success in otolaryngology","volume":"202","author":"K A Daly","year":"2006","journal-title":"J Am Coll Surg"},{"issue":"03","key":"ref28","doi-asserted-by":"crossref","first-page":"256","DOI":"10.1001\/archpedi.154.3.256","article-title":"Information collected during the residency match process does not predict clinical performance","volume":"154","author":"S M Borowitz","year":"2000","journal-title":"Arch Pediatr Adolesc Med"},{"issue":"03","key":"ref29","first-page":"671","article-title":"Orthopaedic in-training examination scores: a correlation with USMLE results","volume":"88","author":"K P Black","year":"2006","journal-title":"J Bone Joint Surg Am"},{"issue":"05","key":"ref30","doi-asserted-by":"crossref","first-page":"528","DOI":"10.1097\/01.SMJ.0000157560.75496.CB","article-title":"Relation of residency selection factors to subsequent orthopaedic in-training examination performance","volume":"98","author":"K D Carmichael","year":"2005","journal-title":"South Med J"},{"issue":"08","key":"ref31","doi-asserted-by":"crossref","first-page":"603","DOI":"10.1016\/S1076-6332(00)80575-9","article-title":"Are NBME examination scores useful in selecting radiology residency candidates?","volume":"7","author":"R B Gunderman","year":"2000","journal-title":"Acad Radiol"},{"issue":"01","key":"ref35","doi-asserted-by":"crossref","first-page":"35","DOI":"10.1111\/j.1540-6261.2010.01625.x","article-title":"When is a liability not a liability? Textual analysis, dictionaries, and 10-Ks","volume":"66","author":"T Loughran","year":"2011","journal-title":"J Finance"},{"issue":"03","key":"ref36","doi-asserted-by":"crossref","first-page":"436","DOI":"10.1111\/j.1467-8640.2012.00460.x","article-title":"Crowdsourcing a word-emotion association lexicon","volume":"29","author":"S Mohammad","year":"2013","journal-title":"Comput Intell"},{"issue":"01","key":"ref37","doi-asserted-by":"crossref","first-page":"476","DOI":"10.1186\/s12909-019-1910-6","article-title":"Gender-based differences in letters of recommendation written for ophthalmology residency applicants","volume":"19","author":"F Lin","year":"2019","journal-title":"BMC Med Educ"},{"issue":"03","key":"ref38","doi-asserted-by":"crossref","first-page":"866","DOI":"10.1016\/j.jsurg.2020.08.049","article-title":"Gender bias in surgical oncology fellowship recommendation letters: gaining progress","volume":"78","author":"M M Grova","year":"2021","journal-title":"J Surg Educ"},{"issue":"07","key":"ref39","doi-asserted-by":"crossref","first-page":"1400","DOI":"10.1097\/CORR.0000000000001053","article-title":"Are there gender-based differences in language in letters of recommendation to an orthopaedic surgery residency program?","volume":"478","author":"A N Kobayashi","year":"2020","journal-title":"Clin Orthop Relat Res"},{"issue":"06","key":"ref40","doi-asserted-by":"crossref","first-page":"1591","DOI":"10.1037\/a0016539","article-title":"Gender and letters of recommendation for academia: agentic and communal differences","volume":"94","author":"J M Madera","year":"2009","journal-title":"J Appl Psychol"},{"issue":"02","key":"ref41","doi-asserted-by":"crossref","first-page":"191","DOI":"10.1177\/0957926503014002277","article-title":"Exploring the color of glass: letters of recommendation for female and male medical faculty","volume":"14","author":"F Trix","year":"2003","journal-title":"Discourse Soc"},{"issue":"03","key":"ref42","doi-asserted-by":"crossref","first-page":"e20","DOI":"10.2106\/JBJS.OA.20.00023","article-title":"Race- and gender-based differences in descriptions of applicants in the letters of recommendation for orthopaedic surgery residency","volume":"5","author":"A Powers","year":"2020","journal-title":"JBJS Open Access"},{"issue":"01","key":"ref43","doi-asserted-by":"crossref","first-page":"56","DOI":"10.1016\/j.urology.2019.05.065","article-title":"The presence of gender bias in letters of recommendations written for urology residency applicants","volume":"134","author":"P Filippou","year":"2019","journal-title":"Urology"},{"issue":"02","key":"ref44","doi-asserted-by":"crossref","first-page":"164","DOI":"10.1016\/j.survophthal.2007.12.007","article-title":"Re-engineering the resident applicant selection process in ophthalmology: a literature review and recommendations for improvement","volume":"53","author":"A G Lee","year":"2008","journal-title":"Surv Ophthalmol"},{"issue":"06","key":"ref45","doi-asserted-by":"crossref","first-page":"391","DOI":"10.1016\/j.cursur.2006.06.012","article-title":"Is it appropriate to use core clerkship grades in the selection of residents?","volume":"63","author":"H Takayama","year":"2006","journal-title":"Curr Surg"}],"container-title":["Journal of Academic Ophthalmology"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/www.thieme-connect.de\/products\/ejournals\/pdf\/10.1055\/s-0041-1733932.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,7,24]],"date-time":"2025-07-24T22:56:51Z","timestamp":1753397811000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.aupojournal.org\/jao\/vol13\/iss2\/6"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,7]]},"references-count":39,"journal-issue":{"issue":"02","published-online":{"date-parts":[[2021,11,10]]},"published-print":{"date-parts":[[2021,7]]}},"URL":"https:\/\/doi.org\/10.1055\/s-0041-1733932","archive":["Portico","CLOCKSS"],"relation":{},"ISSN":["2475-4757"],"issn-type":[{"value":"2475-4757","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,7]]}}}