{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,21]],"date-time":"2026-09-21T23:59:28Z","timestamp":1790035168649,"version":"4.0.1"},"reference-count":20,"publisher":"Ovid Technologies (Wolters Kluwer Health)","funder":[{"name":"None"}],"content-domain":{"domain":["lww.com","ovid.com"],"crossmark-restriction":true},"short-container-title":[],"abstract":"<jats:sec>\n                    <jats:title>Introduction:<\/jats:title>\n                    <jats:p>Physician trainees are increasingly expected to demonstrate leadership in complex, team-based clinical settings. However, assessing leadership performance in dynamic environments like simulation remains challenging, with limited consensus on valid, reliable standards. While some observational frameworks exist, few offer defensible methods for determining minimum passing thresholds, particularly in high-stakes contexts. Traditional test-centered approaches may not capture the behavioral complexity of leadership, whereas examinee-centered methods offer more context-sensitive evaluations. This study compares test-centered and examinee-centered standard-setting approaches to identify a fair, reproducible method for assessing leadership in simulation, offering a model that can be applied across medical education programs.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Methods:<\/jats:title>\n                    <jats:p>This study had 3 phases: first, the extended Angoff method (a test-centered method) was used to identify characteristics of borderline performers to set a cut-score; then we used the borderline group method (an examinee-centered method) to identify the cut-score; and finally, the cut-score was used to identify failing performance in a new data set.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results:<\/jats:title>\n                    <jats:p>The expert panel in phase 1 determined 10.34 as the cut-score; therefore, a score of \u226511 would be required to pass the evaluation. Analyses conducted in phase 2 suggested that 13 should be the cut-score. The mean and SD for phase 3 were similar to those of phase 2, lending support for the cut score of 13.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusions:<\/jats:title>\n                    <jats:p>Setting a cut-score is essential to accurately evaluating student performance when observations and ratings are more subjective. Ultimately, we sought to compare the test-centered versus examinee-centered methods and conclude that the examinee-centered method was a more holistic, fair, and reliable approach to student assessment.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.1097\/sih.0000000000000970","type":"journal-article","created":{"date-parts":[[2026,9,21]],"date-time":"2026-09-21T23:00:14Z","timestamp":1790031614000},"update-policy":"https:\/\/doi.org\/10.1097\/lww.0000000000001000","source":"Crossref","is-referenced-by-count":0,"title":["Improving Medical Student Leadership Evaluation Through Standard Setting\u200b\u200b\u200b\u200b\u200b"],"prefix":"10.1097","author":[{"ORCID":"https:\/\/orcid.org\/0009-0005-6264-6629","authenticated-orcid":false,"given":"Angela M.","family":"Yarnell","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0788-7153","authenticated-orcid":false,"given":"Erin S.","family":"Barry","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2628-8658","authenticated-orcid":false,"given":"Rebekah","family":"Cole","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2637-5129","authenticated-orcid":false,"given":"Dina","family":"Kurzweil","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0003-5515-3354","authenticated-orcid":false,"given":"Karen","family":"Marcellas","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"James","family":"Schwartz","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9252-9723","authenticated-orcid":false,"given":"Leslie","family":"Vojta","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"276","published-online":{"date-parts":[[2026,9,21]]},"reference":[{"issue":"4","key":"R1-20260921","doi-asserted-by":"crossref","first-page":"452","DOI":"10.1108\/LHS-06-2016-0026","article-title":"Leadership in interprofessional health and social care teams: a literature review","volume":"31","author":"Smith","year":"2018","journal-title":"Leadersh Health Serv"},{"issue":"4","key":"R2-20260921","doi-asserted-by":"crossref","first-page":"433","DOI":"10.1037\/amp0000298","article-title":"Teamwork in healthcare: key discoveries enabling safer, high-quality care","volume":"73","author":"Rosen","year":"2018","journal-title":"Am Psychol"},{"issue":"1","key":"R4-20260921","first-page":"64","article-title":"Towards better assessment of leadership in medical trainees","volume":"38","author":"Voyer","year":"2016","journal-title":"Med Teach"},{"issue":"suppl_3","key":"R5-20260921","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1093\/milmed\/usad108","article-title":"Operation Bushmaster: the impact of simulation-based education on preparing the next generation of military medical officers","volume":"188","author":"Rudinsky","year":"2023","journal-title":"Mil Med"},{"issue":"suppl_3","key":"R6-20260921","doi-asserted-by":"crossref","first-page":"15","DOI":"10.1093\/milmed\/usac377","article-title":"Students\u2019 leadership development during a high-fidelity military medical field practicum","volume":"188","author":"Van Shufflin","year":"2023","journal-title":"Mil Med"},{"issue":"11-12","key":"R7-20260921","doi-asserted-by":"crossref","first-page":"653","DOI":"10.1093\/milmed\/usz121","article-title":"Medical student leader performance in an applied medical field practicum","volume":"184","author":"Barry","year":"2019","journal-title":"Mil Med"},{"issue":"3-4","key":"R8-20260921","doi-asserted-by":"crossref","first-page":"358","DOI":"10.1093\/milmed\/usz364","article-title":"Faculty assessments in a military medical field exercise: rater experience and gender do not appear to influence scoring","volume":"185","author":"Barry","year":"2020","journal-title":"Mil Med"},{"issue":"suppl_3","key":"R9-20260921","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1093\/milmed\/usad108","article-title":"Operation Bushmaster: the impact of immersive simulation\u2013based education on preparing the next generation of military medical officers","volume":"188","author":"Rudinsky","year":"2023","journal-title":"Mil Med"},{"key":"R10-20260921","doi-asserted-by":"crossref","first-page":"233","DOI":"10.1007\/s10459-013-9458-4","article-title":"What counts as validity evidence? Examples and prevalence in a systematic review of simulation-based assessment","volume":"19","author":"Cook","year":"2014","journal-title":"Adv Health Sci Educ Theory Pract"},{"key":"R11-20260921","doi-asserted-by":"crossref","first-page":"31","DOI":"10.1186\/s41077-016-0033-y","article-title":"Validation of educational assessments: a primer for simulation and beyond","volume":"1","author":"Cook","year":"2016","journal-title":"Adv Simul"},{"issue":"6","key":"R13-20260921","doi-asserted-by":"crossref","first-page":"560","DOI":"10.1111\/medu.12678","article-title":"Contemporary approach to validity arguments: a practical guide to Kane\u2019s framework","volume":"49","author":"Cook","year":"2015","journal-title":"Med Educ"},{"issue":"3","key":"R14-20260921","doi-asserted-by":"crossref","first-page":"129","DOI":"10.1207\/s15326977ea0503_1","article-title":"Choosing between examinee-centered and test-centered standard-setting methods","volume":"5","author":"Kane","year":"1998","journal-title":"Educ Assess"},{"issue":"7-8","key":"R17-20260921","doi-asserted-by":"crossref","first-page":"e1391","DOI":"10.1093\/milmed\/usae549","article-title":"Gathering validity evidence for the Bushmaster Assessment Tool","volume":"190","author":"Elkins","year":"2024","journal-title":"Mil Med"},{"issue":"suppl","key":"R18-20260921","doi-asserted-by":"crossref","first-page":"87","DOI":"10.1093\/milmed\/usac110","article-title":"Undergraduate medical education leader performance predicts post graduate military leader performance","volume":"188","author":"Barry","year":"2023","journal-title":"Mil Med"},{"key":"R19-20260921","doi-asserted-by":"crossref","first-page":"41","DOI":"10.1207\/s15324818ame0801_4","article-title":"Using an extended Angoff procedure to set standards on complex performance assessments","volume":"8","author":"Hambleton","year":"1995","journal-title":"Appl Meas Educ"},{"issue":"7","key":"R20-20260921","doi-asserted-by":"crossref","first-page":"1079","DOI":"10.1097\/ACM.0000000000002182","article-title":"A comparison of approaches for mastery learning standard setting","volume":"93","author":"Barsuk","year":"2018","journal-title":"Acad Med"},{"issue":"3","key":"R21-20260921","doi-asserted-by":"crossref","first-page":"162","DOI":"10.1111\/j.1600-0579.2008.00568.x","article-title":"Who will pass the dental OSCE? Comparison of the Angoff and the borderline regression standard setting methods","volume":"13","author":"Schoonheim-Klein","year":"2009","journal-title":"Eur J Dent Educ"},{"issue":"5","key":"R22-20260921","doi-asserted-by":"crossref","first-page":"617","DOI":"10.1007\/s10459-007-9068-0","article-title":"Undesired variance due to examiner stringency\/leniency effect in communication skill scores assessed in OSCEs","volume":"13","author":"Harasym","year":"2008","journal-title":"Adv Health Sci Educ Theory Pract"},{"key":"R23-20260921","doi-asserted-by":"crossref","first-page":"71","DOI":"10.1186\/s12909-017-0908-1","article-title":"Examiner effect on the objective structured clinical exam\u2014a study at five medical schools","volume":"17","author":"Schleicher","year":"2017","journal-title":"BMC Med Educ"},{"issue":"3","key":"R24-20260921","doi-asserted-by":"crossref","first-page":"301","DOI":"10.4300\/JGME-D-17-00781.1","article-title":"Effect of trainee performance data on standard-setting judgments using the mastery Angoff method","volume":"10","author":"Prenner","year":"2018","journal-title":"J Grad Med Educ"}],"container-title":["Simulation in Healthcare: The Journal of the Society for Simulation in Healthcare"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.ovid.com\/10.1097\/SIH.0000000000000970","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,9,21]],"date-time":"2026-09-21T23:00:17Z","timestamp":1790031617000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.ovid.com\/10.1097\/SIH.0000000000000970"}},"subtitle":["A Comparison of Test Centered Versus Examinee Centered Methods of Standard Setting"],"short-title":[],"issued":{"date-parts":[[2026,9,21]]},"references-count":20,"URL":"https:\/\/doi.org\/10.1097\/sih.0000000000000970","relation":{},"ISSN":["1559-2332","1559-713X"],"issn-type":[{"value":"1559-2332","type":"print"},{"value":"1559-713X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,9,21]]},"assertion":[{"value":"2024-09-23","name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-07-10","name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}}]}}