{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,1]],"date-time":"2026-09-01T20:13:31Z","timestamp":1788293611369,"version":"build-2803163510"},"reference-count":44,"publisher":"JMIR Publications Inc.","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["JMIR Med Educ"],"abstract":"Abstract<\/jats:title>\n \n Background<\/jats:title>\n The Medical College Admission Test (MCAT) has been central to medical school admissions in North America, though its necessity in holistic selection processes remains debated. The COVID-19 pandemic\u2019s suspension of MCAT testing sessions allowed institutions to explore alternative admission criteria. Additionally, global challenges in test administration underscore the vulnerability of systems dependent on a single standardized test.<\/jats:p>\n <\/jats:sec>\n \n Objective<\/jats:title>\n This study aimed to (1) assess whether including MCAT scores in admissions decisions improves the prediction of medical school performance compared with grade point average (GPA) and interview-based selection and (2) evaluate whether machine learning (ML) can generate viable MCAT score predictions when testing is unavailable.<\/jats:p>\n <\/jats:sec>\n \n Methods<\/jats:title>\n We conducted a retrospective cohort study of 1898 applicants to the Lebanese American University School of Medicine (2009\u20102023). Among 349 admitted students with MCAT scores, we compared admission composites including vs excluding the MCAT in relation to academic outcomes (Med 1\u20104 final grades) and clinical performance (Med 1\u20102 objective structured clinical examination [OSCE] scores). Students were stratified into tertiles to examine tier-specific effects. To model an MCAT alternative, we trained an ensemble ML model (least absolute shrinkage and selection operator, kernel ridge, gradient boosting, elastic net, and light gradient boosting machine [LightGBM]) using data from 1583 applicants (2009\u20102020) to predict MCAT scores from cumulative GPA, core GPA, interview scores, merit points, and honors participation. The model was validated on 315 applicants admitted during the MCAT suspension (2021\u20102023), and its impact on admission rankings and performance correlations was evaluated.<\/jats:p>\n <\/jats:sec>\n \n Results<\/jats:title>\n \n Including the MCAT in the admissions composite did not meaningfully improve prediction of overall academic performance (\n r<\/jats:italic>\n =0.63, 95% CI 0.56-0.69 with MCAT vs\n r<\/jats:italic>\n =0.62, 95% CI 0.55-0.68 without MCAT;\n P<\/jats:italic>\n =.63). However, it significantly weakened prediction of clinical skills (OSCE:\n r<\/jats:italic>\n =0.37, 95% CI 0.27-0.46 with MCAT vs\n r<\/jats:italic>\n =0.48, 95% CI 0.40-0.56 without MCAT;\n P<\/jats:italic>\n <.001). In tertile analyses, the MCAT modestly improved prediction among top-performing students but eliminated predictive validity in the bottom tertile (\n r<\/jats:italic>\n =0.12, 95% CI \u20130.07 to 0.30;\n P<\/jats:italic>\n =.21 vs\n r<\/jats:italic>\n =0.28, 95% CI 0.10-0.44 without MCAT;\n P<\/jats:italic>\n =.003). The ML model explained 36% of MCAT variance (\n R<\/jats:italic>\n \u00b2=0.36 with 95% CI 0.32-0.41; root mean squared error \u224810% of score range). Incorporating predicted MCAT scores reduced correlations with subsequent performance across all outcomes. Although rankings based on predicted MCAT scores strongly correlated with original rankings (\n r<\/jats:italic>\n =0.80, 95% CI 0.77-0.83), 6.3% (4\/64) to 9.4% (6\/64) of admission decisions would have changed.\n <\/jats:p>\n <\/jats:sec>\n \n Conclusions<\/jats:title>\n Within this single-institution study, MCAT scores provided limited incremental validity for predicting academic performance and reduced prediction of clinical skills. ML-based predictions of MCAT scores introduced sufficient error to affect admissions decisions, supporting a robust holistic admissions process during temporary MCAT disruptions while highlighting the need for validation across diverse institutional settings before broader generalization.<\/jats:p>\n <\/jats:sec>","DOI":"10.2196\/94197","type":"journal-article","created":{"date-parts":[[2026,9,1]],"date-time":"2026-09-01T19:15:10Z","timestamp":1788290110000},"page":"e94197-e94197","source":"Crossref","is-referenced-by-count":0,"title":["Evaluating the Influence of MCAT Scores on Medical Student Selection and Performance Using a Machine Learning Approach: Retrospective Cohort Study"],"prefix":"10.2196","volume":"12","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-6438-0159","authenticated-orcid":false,"given":"Michele","family":"Cherfane","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7479-7994","authenticated-orcid":false,"given":"Marc","family":"Ghanem","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1456-9292","authenticated-orcid":false,"given":"Georges","family":"Choueiry","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9334-9847","authenticated-orcid":false,"given":"Selim M","family":"Nasser","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3666-8312","authenticated-orcid":false,"given":"Tamina","family":"Elias-Rizk","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7159-7559","authenticated-orcid":false,"given":"Sola","family":"Aoun Bahous","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1010","published-online":{"date-parts":[[2026,9,1]]},"reference":[{"issue":"21","key":"R1","doi-asserted-by":"publisher","first-page":"2250","DOI":"10.1001\/jama.2012.74126","article-title":"Transforming admissions: the gateway to medicine","volume":"308","author":"Kirch","journal-title":"JAMA"},{"issue":"4","key":"R2","doi-asserted-by":"publisher","first-page":"376","DOI":"10.1111\/medu.13482","article-title":"What do we know about coaching in medical education? A literature review","volume":"52","author":"Lovell","journal-title":"Med Educ"},{"issue":"1","key":"R3","doi-asserted-by":"publisher","DOI":"10.1186\/s12909-018-1316-x","article-title":"Selection into medicine: the predictive validity of an outcome-based procedure","volume":"18","author":"Schreurs","journal-title":"BMC Med Educ"},{"issue":"5","key":"R4","doi-asserted-by":"publisher","first-page":"666","DOI":"10.1097\/ACM.0b013e3182864299","article-title":"The predictive validity of the MCAT exam in relation to academic performance through medical school: a national cohort study of 2001-2004 matriculants","volume":"88","author":"Dunleavy","journal-title":"Acad Med"},{"issue":"21","key":"R5","doi-asserted-by":"publisher","first-page":"2243","DOI":"10.1001\/jama.2013.282093","article-title":"The new 2015 MCAT: testing competencies","volume":"310","author":"Kirch","journal-title":"JAMA"},{"issue":"1","key":"R6","doi-asserted-by":"publisher","first-page":"100","DOI":"10.1097\/01.ACM.0000249878.25186.b7","article-title":"The predictive validity of the MCAT for medical school performance and medical board licensing examinations: a meta-analysis of the published research","volume":"82","author":"Donnon","journal-title":"Acad Med"},{"issue":"3","key":"R7","doi-asserted-by":"publisher","first-page":"387","DOI":"10.1097\/ACM.0000000000002942","article-title":"The validity of scores from the new MCAT exam in predicting student performance: results from a multisite study","volume":"95","author":"Busche","journal-title":"Acad Med"},{"issue":"3","key":"R8","doi-asserted-by":"publisher","first-page":"396","DOI":"10.1097\/ACM.0000000000003064","article-title":"A study of the validity of the new MCAT exam","volume":"95","author":"Violato","journal-title":"Acad Med"},{"issue":"1","key":"R9","doi-asserted-by":"publisher","DOI":"10.1186\/s12909-016-0652-y","article-title":"Predictors of medical school clerkship performance: a multispecialty longitudinal analysis of standardized examination scores and clinical assessments","volume":"16","author":"Casey","journal-title":"BMC Med Educ"},{"issue":"5","key":"R10","doi-asserted-by":"publisher","first-page":"572","DOI":"10.1097\/ACM.0b013e31828b85af","article-title":"Should the MCAT exam be used for medical school admissions in Canada?","volume":"88","author":"Eskander","journal-title":"Acad Med"},{"issue":"5815","key":"R11","doi-asserted-by":"publisher","first-page":"1080","DOI":"10.1126\/science.1136618","article-title":"Assessment. Standardized tests predict graduate students\u2019 success","volume":"315","author":"Kuncel","journal-title":"Science"},{"issue":"10","key":"R12","doi-asserted-by":"publisher","first-page":"910","DOI":"10.1097\/00001888-200510000-00010","article-title":"Validity of the Medical College Admission Test for predicting medical school performance","volume":"80","author":"Julian","journal-title":"Acad Med"},{"issue":"7","key":"R13","doi-asserted-by":"publisher","first-page":"953","DOI":"10.1097\/ACM.0000000000000653","article-title":"The role of socioeconomic status in medical school admissions: validation of a socioeconomic indicator for use in medical school admissions","volume":"90","author":"Grbic","journal-title":"Acad Med"},{"issue":"1","key":"R14","doi-asserted-by":"publisher","DOI":"10.1186\/s12909-026-08940-2","article-title":"Socioeconomic background, exam performance, and probability of medical school application among all MCAT examinees, 2017 to 2019","volume":"26","author":"Shahriar","journal-title":"BMC Med Educ"},{"issue":"3","key":"R15","doi-asserted-by":"publisher","first-page":"282","DOI":"10.1177\/1745691615577701","article-title":"One century of global IQ gains: a formal meta-analysis of the Flynn effect (1909-2013)","volume":"10","author":"Pietschnig","journal-title":"Perspect Psychol Sci"},{"issue":"5","key":"R16","doi-asserted-by":"publisher","first-page":"1332","DOI":"10.1037\/a0037173","article-title":"The Flynn effect: a meta-analysis","volume":"140","author":"Trahan","journal-title":"Psychol Bull"},{"issue":"17","key":"R17","doi-asserted-by":"publisher","first-page":"1565","DOI":"10.1056\/NEJMp1300411","article-title":"Holistic review--shaping the medical profession one applicant at a time","volume":"368","author":"Witzburg","journal-title":"N Engl J Med"},{"issue":"1","key":"R18","doi-asserted-by":"publisher","first-page":"36","DOI":"10.1111\/medu.12817","article-title":"How effective are selection methods in medical education? A systematic review","volume":"50","author":"Patterson","journal-title":"Med Educ"},{"key":"R19","doi-asserted-by":"publisher","DOI":"10.1177\/23821205211014898","article-title":"Admissions to medical school during the COVID-19 era without the MCAT","volume":"8","author":"Corridon","journal-title":"J Med Educ Curric Dev"},{"issue":"2","key":"R20","doi-asserted-by":"publisher","first-page":"196","DOI":"10.1016\/j.jemermed.2015.02.008","article-title":"Prognostic value of the multiple mini-interview for emergency medicine residency performance","volume":"49","author":"Burkhardt","journal-title":"J Emerg Med"},{"issue":"1","key":"R21","doi-asserted-by":"publisher","first-page":"47","DOI":"10.1001\/jama.1987.03390010051027","article-title":"Medical school performance of initially rejected students","volume":"257","author":"DeVaul","journal-title":"JAMA"},{"issue":"2","key":"R22","doi-asserted-by":"publisher","first-page":"161","DOI":"10.1023\/B:AHSE.0000027479.14637.6c","article-title":"Where judgement fails: pitfalls in the selection process for medical personnel","volume":"9","author":"Eva","journal-title":"Adv Health Sci Educ Theory Pract"},{"issue":"1","key":"R23","doi-asserted-by":"publisher","DOI":"10.1186\/s12909-014-0267-0","article-title":"The fairness, predictive validity and acceptability of multiple mini interview in an internationally diverse student population--a mixed methods study","volume":"14","author":"Kelly","journal-title":"BMC Med Educ"},{"issue":"2","key":"R24","doi-asserted-by":"publisher","first-page":"147","DOI":"10.1023\/B:AHSE.0000027464.22411.0f","article-title":"Investigating the reliability of the medical school admissions interview","volume":"9","author":"Kreiter","journal-title":"Adv Health Sci Educ Theory Pract"},{"issue":"2","key":"R25","doi-asserted-by":"publisher","first-page":"159","DOI":"10.1023\/a:1011489618208","article-title":"Reliability and validity of admissions tools used to select students for the health professions","volume":"6","author":"Salvatori","journal-title":"Adv Health Sci Educ Theory Pract"},{"issue":"6","key":"R26","doi-asserted-by":"publisher","first-page":"532","DOI":"10.1097\/00001888-199506000-00015","article-title":"Influence of medical school applicants' demographic and cognitive characteristics on interviewers' ratings of noncognitive traits","volume":"70","author":"Shaw","journal-title":"Acad Med"},{"issue":"1","key":"R27","doi-asserted-by":"publisher","first-page":"151","DOI":"10.1007\/s10459-017-9779-9","article-title":"The influence of MCAT and GPA preadmission academic metrics on interview scores","volume":"23","author":"Gay","journal-title":"Adv Health Sci Educ Theory Pract"},{"issue":"1","key":"R28","doi-asserted-by":"publisher","first-page":"112","DOI":"10.1093\/bioinformatics\/btr597","article-title":"MissForest--non-parametric missing value imputation for mixed-type data","volume":"28","author":"Stekhoven","journal-title":"Bioinformatics"},{"issue":"9","key":"R29","doi-asserted-by":"publisher","first-page":"1374","DOI":"10.1097\/ACM.0000000000004754","article-title":"The validity of MCAT scores in predicting students\u2019 performance and progress in medical school: results from a multisite study","volume":"97","author":"Hanson","journal-title":"Acad Med"},{"key":"R30","unstructured":"MCAT validity data report. Association of American Medical Colleges (AAMC); 2024. URL: https:\/\/www.aamc.org\/media\/78266\/download [Accessed 13-08-2026]"},{"issue":"6","key":"R31","doi-asserted-by":"publisher","first-page":"479","DOI":"10.1016\/j.jnma.2025.08.108","article-title":"Forecasting medical school success: a systematic review and meta-analysis of the predictive value of MCAT scores","volume":"117","author":"Harvey","journal-title":"J Natl Med Assoc"},{"issue":"Suppl 2","key":"R32","doi-asserted-by":"publisher","first-page":"7","DOI":"10.1093\/milmed\/usad010","article-title":"Blinding of admissions committee to MCAT scores: is there a difference in subsequent medical school performance between cohorts?","volume":"188","author":"Teng","journal-title":"Mil Med"},{"issue":"10 Suppl","key":"R33","doi-asserted-by":"publisher","first-page":"S60","DOI":"10.1097\/ACM.0b013e3181ed442b","article-title":"The reliability and acceptability of the multiple mini-interview as a selection instrument for postgraduate admissions","volume":"85","author":"Dore","journal-title":"Acad Med"},{"issue":"4 Suppl","key":"R34","doi-asserted-by":"publisher","first-page":"4","DOI":"10.7205\/MILMED-D-14-00550","article-title":"Does the MCAT predict medical school and PGY-1 performance?","volume":"180","author":"Saguil","journal-title":"Mil Med"},{"key":"R35","doi-asserted-by":"publisher","DOI":"10.3402\/meo.v21.31795","article-title":"Do MCAT scores predict USMLE scores? An analysis on 5 years of medical student data","volume":"21","author":"Gauer","journal-title":"Med Educ Online"},{"issue":"6","key":"R36","doi-asserted-by":"publisher","first-page":"980","DOI":"10.1097\/ACM.0b013e3181cece3d","article-title":"The predictive validity of three versions of the MCAT in relation to performance in medical school, residency, and licensing examinations: a longitudinal study of 36 classes of Jefferson Medical College","volume":"85","author":"Callahan","journal-title":"Acad Med"},{"issue":"9","key":"R37","doi-asserted-by":"publisher","first-page":"e1437","DOI":"10.3109\/0142159X.2013.818634","article-title":"The objective structured clinical examination (OSCE): AMEE guide no. 81. Part I: an historical and theoretical perspective","volume":"35","author":"Khan","journal-title":"Med Teach"},{"issue":"3","key":"R38","doi-asserted-by":"publisher","first-page":"213","DOI":"10.1177\/01632787231165797","article-title":"Comparability of objective structured clinical examinations (OSCEs) and written tests for assessing medical school students\u2019 competencies: a scoping review","volume":"46","author":"Chang","journal-title":"Eval Health Prof"},{"issue":"3","key":"R39","doi-asserted-by":"publisher","first-page":"314","DOI":"10.1046\/j.1365-2923.2004.01776.x","article-title":"An admissions OSCE: the multiple mini-interview","volume":"38","author":"Eva","journal-title":"Med Educ"},{"issue":"12","key":"R40","doi-asserted-by":"publisher","first-page":"1027","DOI":"10.3109\/0142159X.2013.829912","article-title":"The multiple mini-interview (MMI) for student selection in health professions training - a systematic review","volume":"35","author":"Pau","journal-title":"Med Teach"},{"issue":"5","key":"R41","doi-asserted-by":"publisher","first-page":"532","DOI":"10.1037\/a0015808","article-title":"An effect size primer: a guide for clinicians and researchers","volume":"40","author":"Ferguson","journal-title":"Prof Psychol Res Pract"},{"issue":"9","key":"R42","doi-asserted-by":"publisher","first-page":"1036","DOI":"10.1097\/ACM.0000000000005202","article-title":"Artificial intelligence screening of medical school applications: development and validation of a machine-learning algorithm","volume":"98","author":"Triola","journal-title":"Acad Med"},{"issue":"1","key":"R43","doi-asserted-by":"publisher","DOI":"10.1093\/jamiaopen\/ooad011","article-title":"Using artificial intelligence in medical school admissions screening to decrease inter- and intra-observer variability","volume":"6","author":"Keir","journal-title":"JAMIA Open"},{"issue":"3","key":"R44","doi-asserted-by":"publisher","DOI":"10.1007\/s42979-021-00592-x","article-title":"Machine learning: algorithms, real-world applications and research directions","volume":"2","author":"Sarker","journal-title":"SN Comput Sci"}],"container-title":["JMIR Medical Education"],"original-title":[],"language":"en","deposited":{"date-parts":[[2026,9,1]],"date-time":"2026-09-01T19:15:11Z","timestamp":1788290111000},"score":1,"resource":{"primary":{"URL":"https:\/\/mededu.jmir.org\/2026\/1\/e94197"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,9,1]]},"references-count":44,"URL":"https:\/\/doi.org\/10.2196\/94197","relation":{},"ISSN":["2369-3762"],"issn-type":[{"value":"2369-3762","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,9,1]]},"article-number":"v12i21e94197"}}