{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,2]],"date-time":"2026-02-02T04:38:42Z","timestamp":1770007122375,"version":"3.49.0"},"reference-count":22,"publisher":"SAGE Publications","license":[{"start":{"date-parts":[[2026,1,1]],"date-time":"2026-01-01T00:00:00Z","timestamp":1767225600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"},{"start":{"date-parts":[[2026,1,1]],"date-time":"2026-01-01T00:00:00Z","timestamp":1767225600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/journals.sagepub.com\/page\/policies\/text-and-data-mining-license"}],"content-domain":{"domain":["journals.sagepub.com"],"crossmark-restriction":true},"short-container-title":["Journal of Medical Education and Curricular Development"],"published-print":{"date-parts":[[2026,1]]},"abstract":"<jats:sec>\n                    <jats:title>Background<\/jats:title>\n                    <jats:p>Multiple-mini-interviews (MMIs) are the most commonly used non-academic assessment tool for British medical school admissions processes. Potential inconsistencies can arise from running MMIs, such as differing marking standards among interviewers and stations with varying levels of difficulty.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Methods<\/jats:title>\n                    <jats:p>With the aim of analysing MMI data, the cumulative probit mixed model was deployed which accounts for latent sources of variation inherent to MMI scores \u2013 both external factors, such as interviewer behaviour and station complexity, as well as the factor of interest \u2013 candidates\u2019 true performance at interview. With the secondary aim of making this methodology more accessible to non-statistical experts, we developed a user-friendly application using R Shiny. The app was created to standardise MMI scores and generate feedback for interviewers without requiring prior knowledge of programming.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>\n                      MMI data from Lancaster Medical School were analysed for the academic year 2022\u20132023. Applicant ability (\n                      <jats:italic toggle=\"yes\">n<\/jats:italic>\n                      \u2009=\u2009352) contributed to 22.94% of the total variance in MMI scores. Notably, interviewers (\n                      <jats:italic toggle=\"yes\">n<\/jats:italic>\n                      \u2009=\u200983) contributed a smaller proportion of variance (10.79%). Station difficulty (\n                      <jats:italic toggle=\"yes\">n<\/jats:italic>\n                      \u2009=\u20099) had a minor impact on variance (2.23%), with inter-station reliability being acceptable based on the Cronbach's alpha value (\u03b1\u2009=\u20090.7072).\n                    <\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusion<\/jats:title>\n                    <jats:p>The current method provides a statistically robust approach towards the analysis of MMI scores. In addition, its companion application can be used by admissions staff to communicate feedback to interviewers on their scoring patterns and identify stations that are better at discriminating applicants\u2019 ability. The illustrated approach can be adapted for use by other medical institutions that use MMI scores to rank candidates during their admissions processes. Future research could investigate how to extend the proposed modelling approach to address applicants\u2019 background factors, such as socio-economic status, for more comprehensive analysis and more equitable offer allocation.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.1177\/23821205251411170","type":"journal-article","created":{"date-parts":[[2026,1,30]],"date-time":"2026-01-30T17:39:38Z","timestamp":1769794778000},"update-policy":"https:\/\/doi.org\/10.1177\/sage-journals-update-policy","source":"Crossref","is-referenced-by-count":0,"title":["Utilising Random Effects Models to Analyse Multiple Mini-Interviews for Prospective Medical Students \u2013 From Theory to Practice"],"prefix":"10.1177","volume":"13","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-0120-2141","authenticated-orcid":false,"given":"Chezko Malachi Peligrino","family":"Castro","sequence":"first","affiliation":[{"name":"Lancaster University Medical School"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3056-0557","authenticated-orcid":false,"given":"Nicola","family":"Phillips","sequence":"additional","affiliation":[{"name":"Lancaster University Medical School"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Karen","family":"Grant","sequence":"additional","affiliation":[{"name":"Lancaster University Medical School"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3770-9710","authenticated-orcid":false,"given":"Iain","family":"Robinson","sequence":"additional","affiliation":[{"name":"Lancaster University Medical School"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0640-181X","authenticated-orcid":false,"given":"Emanuele","family":"Giorgi","sequence":"additional","affiliation":[{"name":"Lancaster Medical School, Centre for Health Informatics Computing and Statistics, Lancaster University"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"179","published-online":{"date-parts":[[2026,1,30]]},"reference":[{"key":"e_1_3_2_2_2","unstructured":"NHS England. NHS Long Term Workforce Plan. Accessed February 5 2025. https:\/\/www.england.nhs.uk\/publication\/nhs-long-term-workforce-plan\/."},{"key":"e_1_3_2_3_2","doi-asserted-by":"publisher","DOI":"10.1046\/j.1365-2923.2004.01776.x"},{"key":"e_1_3_2_4_2","unstructured":"Medical Schools Council. Selecting for Excellence: Final Report. Accessed 5 Feb 2025. https:\/\/www.medschools.ac.uk\/latest\/publications\/selecting-for-excellence-final-report\/."},{"key":"e_1_3_2_5_2","doi-asserted-by":"publisher","DOI":"10.1001\/jama.2012.36914"},{"key":"e_1_3_2_6_2","doi-asserted-by":"publisher","DOI":"10.1111\/j.1365-2923.2009.03407.x"},{"key":"e_1_3_2_7_2","doi-asserted-by":"publisher","DOI":"10.1111\/medu.12193"},{"key":"e_1_3_2_8_2","doi-asserted-by":"publisher","DOI":"10.5694\/j.1326-5377.2010.tb03946.x"},{"key":"e_1_3_2_9_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10459-017-9794-x"},{"issue":"2","key":"e_1_3_2_10_2","first-page":"189","article-title":"Detecting and measuring rater effects using many-facet Rasch measurement: Part II","volume":"5","author":"Myford CM","year":"2004","unstructured":"Myford CM, Wolfe EW. Detecting and measuring rater effects using many-facet Rasch measurement: Part II. J Appl Meas. 2004;5(2):189\u2013227. https:\/\/pubmed.ncbi.nlm.nih.gov\/15064538\/. Accessed 5 Feb 2025.","journal-title":"J Appl Meas"},{"key":"e_1_3_2_11_2","doi-asserted-by":"publisher","DOI":"10.1097\/acm.0000000000000650"},{"key":"e_1_3_2_12_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10459-013-9463-7"},{"key":"e_1_3_2_13_2","doi-asserted-by":"publisher","DOI":"10.3109\/0142159X.2012.652706"},{"key":"e_1_3_2_14_2","doi-asserted-by":"publisher","DOI":"10.1080\/10401334.2016.1205997"},{"key":"e_1_3_2_15_2","doi-asserted-by":"publisher","DOI":"10.1046\/j.1365-2648.1994.20010196.x"},{"key":"e_1_3_2_16_2","doi-asserted-by":"publisher","DOI":"10.1002\/9780470594001"},{"key":"e_1_3_2_17_2","unstructured":"Chang W Chang J Allaire J et al. shiny: Web Application Framework for R. 2024. R package version 1.10.0. https:\/\/CRAN.R-project.org\/package=shiny."},{"key":"e_1_3_2_18_2","doi-asserted-by":"publisher","DOI":"10.5093\/psed2022a14"},{"key":"e_1_3_2_19_2","unstructured":"R. Core Team. R: A Language and Environment for Statistical Computing. 2024. https:\/\/www.R-project.org\/."},{"key":"e_1_3_2_20_2","unstructured":"Christensen RHB. Ordinal: Regression Models for Ordinal Data. 2024. R package version 2023.12-4.1. https:\/\/github.com\/runehaubo\/ordinal."},{"key":"e_1_3_2_21_2","doi-asserted-by":"publisher","DOI":"10.1111\/j.1365-2923.2010.03689.x"},{"key":"e_1_3_2_22_2","doi-asserted-by":"publisher","DOI":"10.1097\/ACM.0b013e31827c0c5d"},{"key":"e_1_3_2_23_2","doi-asserted-by":"publisher","DOI":"10.3109\/0142159X.2016.1158799"}],"container-title":["Journal of Medical Education and Curricular Development"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/23821205251411170","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/full-xml\/10.1177\/23821205251411170","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/23821205251411170","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,2,1]],"date-time":"2026-02-01T19:57:30Z","timestamp":1769975850000},"score":1,"resource":{"primary":{"URL":"https:\/\/journals.sagepub.com\/doi\/10.1177\/23821205251411170"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,1]]},"references-count":22,"alternative-id":["10.1177\/23821205251411170"],"URL":"https:\/\/doi.org\/10.1177\/23821205251411170","relation":{},"ISSN":["2382-1205","2382-1205"],"issn-type":[{"value":"2382-1205","type":"print"},{"value":"2382-1205","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,1]]},"article-number":"23821205251411170"}}