{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,1]],"date-time":"2026-05-01T07:11:38Z","timestamp":1777619498023,"version":"3.51.4"},"reference-count":62,"publisher":"SAGE Publications","issue":"1","license":[{"start":{"date-parts":[[2025,10,4]],"date-time":"2025-10-04T00:00:00Z","timestamp":1759536000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"},{"start":{"date-parts":[[2025,10,4]],"date-time":"2025-10-04T00:00:00Z","timestamp":1759536000000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/journals.sagepub.com\/page\/policies\/text-and-data-mining-license"}],"content-domain":{"domain":["journals.sagepub.com"],"crossmark-restriction":true},"short-container-title":["Educational and Psychological Measurement"],"published-print":{"date-parts":[[2026,2]]},"abstract":"<jats:p>In item response theory modeling, item fit analysis using posterior expectations, otherwise known as pseudocounts, has many advantages. They are readily obtained from the E-step output of the Bock\u2013Aitkin Expectation-Maximization (EM) algorithm and continue to function as a basis of evaluating model fit, even when missing data are present. This paper aimed to improve the interpretability of the root mean squared deviation (RMSD) index based on posterior expectations. In Study 1, we assessed its performance using two approaches. First, we employed the poor person\u2019s posterior predictive model checking (PP-PPMC) to compute their significance levels. The resulting Type I error was generally controlled below the nominal level, but power noticeably declined with smaller sample sizes and shorter test lengths. Second, we used receiver operating characteristic (ROC) curve analysis to empirically determine the reference values (cutoff thresholds) that achieve an optimal balance between false-positive and true-positive rates. Importantly, we identified optimal reference values for each combination of sample size and test length in the simulation conditions. The cutoff threshold approach outperformed the PP-PPMC approach with greater gains in true-positive rates than losses from the inflated false-positive rates. In Study 2, we extended the cutoff threshold approach to conditions with larger sample sizes and longer test lengths. Moreover, we evaluated the performance of the optimized cutoff thresholds under varying levels of data missingness. Finally, we employed response surface analysis to develop a prediction model that generalizes the way the reference values vary with sample size and test length. Overall, this study demonstrates the application of the PP-PPMC for item fit diagnostics and implements a practical frequentist approach to empirically derive reference values. Using our prediction model, practitioners can compute the reference values of RMSD that are tailored to their dataset\u2019s sample size and test length.<\/jats:p>","DOI":"10.1177\/00131644251369532","type":"journal-article","created":{"date-parts":[[2025,10,4]],"date-time":"2025-10-04T10:59:08Z","timestamp":1759575548000},"page":"156-190","update-policy":"https:\/\/doi.org\/10.1177\/sage-journals-update-policy","source":"Crossref","is-referenced-by-count":1,"title":["Evaluation of Item Fit With Output From the EM Algorithm: RMSD Index Based on Posterior Expectations"],"prefix":"10.1177","volume":"86","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-1335-2908","authenticated-orcid":false,"given":"Yun-Kyung","family":"Kim","sequence":"first","affiliation":[{"name":"University of California, Los Angeles, CA, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Li","family":"Cai","sequence":"additional","affiliation":[{"name":"University of California, Los Angeles, CA, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"YoungKoung","family":"Kim","sequence":"additional","affiliation":[{"name":"College Board, New York, NY, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"179","published-online":{"date-parts":[[2025,10,4]]},"reference":[{"key":"e_1_3_3_2_1","first-page":"235","volume-title":"The NAEP 1996 Technical Report","author":"Allen N. L.","year":"1999","unstructured":"Allen N. L., Johnson E. G., Mislevy R. J., Thomas N. (1999). Scaling procedures. In National Center for Education Statistics, The NAEP 1996 Technical Report (pp. 235\u2013254), NCES 1999-452. U.S. Department of Education."},{"key":"e_1_3_3_3_1","doi-asserted-by":"publisher","DOI":"10.1214\/aoms\/1177729437"},{"key":"e_1_3_3_4_1","doi-asserted-by":"publisher","DOI":"10.1080\/01621459.1954.10501232"},{"key":"e_1_3_3_5_1","doi-asserted-by":"publisher","DOI":"10.1214\/088342304000000116"},{"key":"e_1_3_3_6_1","doi-asserted-by":"publisher","DOI":"10.1177\/0146621618798666"},{"key":"e_1_3_3_7_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.talanta.2008.05.019"},{"key":"e_1_3_3_8_1","doi-asserted-by":"publisher","DOI":"10.1007\/BF02291411"},{"key":"e_1_3_3_9_1","doi-asserted-by":"publisher","DOI":"10.1007\/BF02293801"},{"key":"e_1_3_3_10_1","doi-asserted-by":"publisher","DOI":"10.2307\/3001663"},{"key":"e_1_3_3_11_1","doi-asserted-by":"publisher","DOI":"10.2307\/2982063"},{"key":"e_1_3_3_12_1","doi-asserted-by":"publisher","DOI":"10.1177\/0146621617748323"},{"key":"e_1_3_3_13_1","doi-asserted-by":"publisher","DOI":"10.1348\/000711007X249603"},{"key":"e_1_3_3_14_1","volume-title":"flexMIRT\u00ae version 3.72: Flexible multilevel multidimensional item analysis and test scoring","author":"Cai L.","year":"2024","unstructured":"Cai L. (2024). flexMIRT\u00ae version 3.72: Flexible multilevel multidimensional item analysis and test scoring [Computer software]. Vector Psychometric Group."},{"key":"e_1_3_3_15_1","volume-title":"Some asymptotic results on the distribution of an IRT measure of item fit","author":"Donoghue J. R.","year":"1999","unstructured":"Donoghue J. R., McClellan C. M. (1999). Some asymptotic results on the distribution of an IRT measure of item fit [Conference presentation]. Annual meeting of the Psychometric Society, Lawrence, Kansas."},{"key":"e_1_3_3_16_1","volume-title":"A corrected asymptotic distribution of an IRT fit measure that accounts for the effects of item parameter estimation","author":"Donoghue J. R.","year":"2003","unstructured":"Donoghue J. R., McClellan C. M. (2003). A corrected asymptotic distribution of an IRT fit measure that accounts for the effects of item parameter estimation [Conference presentation]. Annual meeting of the National Council on Measurement in Education, Chicago, Illinois."},{"key":"e_1_3_3_17_1","doi-asserted-by":"publisher","DOI":"10.1177\/01466210122032046"},{"key":"e_1_3_3_18_1","doi-asserted-by":"publisher","DOI":"10.1002\/ets2.12353"},{"key":"e_1_3_3_19_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11336-014-9428-7"},{"key":"e_1_3_3_20_1","doi-asserted-by":"publisher","DOI":"10.1111\/jedm.12111"},{"issue":"4","key":"e_1_3_3_21_1","first-page":"733","article-title":"Posterior predictive assessment of model fitness via realized discrepancies","volume":"6","author":"Gelman A.","year":"1996","unstructured":"Gelman A., Meng X. L., Stern H. (1996). Posterior predictive assessment of model fitness via realized discrepancies. Statistica Sinica, 6(4), 733\u2013760.","journal-title":"Statistica Sinica"},{"key":"e_1_3_3_22_1","doi-asserted-by":"publisher","DOI":"10.1509\/jmkr.47.5.896"},{"key":"e_1_3_3_23_1","doi-asserted-by":"publisher","DOI":"10.1080\/00949655.2019.1590574"},{"key":"e_1_3_3_24_1","volume-title":"flexMIRT\u00ae user\u2019s manual version 3.7: Flexible multilevel multidimensional item analysis and test scoring","author":"Houts C. R.","year":"2024","unstructured":"Houts C. R., Cai L. (2024). flexMIRT\u00ae user\u2019s manual version 3.7: Flexible multilevel multidimensional item analysis and test scoring. Vector Psychometric Group."},{"key":"e_1_3_3_25_1","doi-asserted-by":"publisher","DOI":"10.1111\/emip.12404"},{"key":"e_1_3_3_26_1","doi-asserted-by":"publisher","DOI":"10.1111\/jedm.12316"},{"key":"e_1_3_3_27_1","doi-asserted-by":"publisher","DOI":"10.3102\/10769986231183326"},{"key":"e_1_3_3_28_1","doi-asserted-by":"publisher","DOI":"10.1111\/j.1745-3984.2008.00071.x"},{"key":"e_1_3_3_29_1","doi-asserted-by":"publisher","DOI":"10.3102\/1076998619890566"},{"key":"e_1_3_3_30_1","doi-asserted-by":"publisher","DOI":"10.1111\/bmsp.12224"},{"key":"e_1_3_3_31_1","doi-asserted-by":"publisher","DOI":"10.1177\/0146621618779985"},{"key":"e_1_3_3_32_1","doi-asserted-by":"publisher","DOI":"10.1080\/10705511.2015.1014041"},{"key":"e_1_3_3_33_1","doi-asserted-by":"publisher","DOI":"10.1177\/0013164408322026"},{"key":"e_1_3_3_34_1","doi-asserted-by":"publisher","DOI":"10.1037\/met0000443"},{"key":"e_1_3_3_35_1","doi-asserted-by":"publisher","DOI":"10.1177\/0146621608329504"},{"key":"e_1_3_3_36_1","unstructured":"Li D. (2005). The asymptotic distribution of an IRT measure for item fit based on pseudocounts [Doctoral dissertation Michigan State University]. ProQuest Dissertations & Theses Global."},{"key":"e_1_3_3_37_1","doi-asserted-by":"publisher","DOI":"10.1198\/000313006X117837"},{"key":"e_1_3_3_38_1","doi-asserted-by":"publisher","DOI":"10.1037\/met0000425"},{"key":"e_1_3_3_39_1","doi-asserted-by":"publisher","DOI":"10.1214\/aos\/1176325622"},{"key":"e_1_3_3_40_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0001-2998(78)80014-2"},{"key":"e_1_3_3_41_1","doi-asserted-by":"publisher","DOI":"10.4097\/kja.21209"},{"key":"e_1_3_3_42_1","doi-asserted-by":"publisher","DOI":"10.1177\/01466216000241003"},{"key":"e_1_3_3_43_1","doi-asserted-by":"publisher","DOI":"10.1177\/0146621603027004004"},{"key":"e_1_3_3_44_1","doi-asserted-by":"publisher","DOI":"10.1093\/oso\/9780198509844.001.0001"},{"key":"e_1_3_3_45_1","doi-asserted-by":"publisher","DOI":"10.1093\/aje\/kwj063"},{"key":"e_1_3_3_46_1","unstructured":"R Core Team (2024). R: A language and environment for statistical computing [Computer software]. R Foundation for Statistical Computing. https:\/\/www.R-project.org\/"},{"key":"e_1_3_3_47_1","doi-asserted-by":"publisher","DOI":"10.3390\/foundations2020032"},{"key":"e_1_3_3_48_1","unstructured":"Robitzsch A. Kiefer T. Wu M. (2024). TAM: Test analysis modules. R package version 4.2-21. https:\/\/CRAN.R-project.org\/package=TAM"},{"key":"e_1_3_3_49_1","doi-asserted-by":"publisher","DOI":"10.1214\/aos\/1176346785"},{"key":"e_1_3_3_50_1","doi-asserted-by":"publisher","DOI":"10.1111\/j.1745-3984.2005.00021.x"},{"key":"e_1_3_3_51_1","doi-asserted-by":"publisher","DOI":"10.1348\/000711005X66888"},{"key":"e_1_3_3_52_1","doi-asserted-by":"publisher","DOI":"10.1177\/0146621605285517"},{"key":"e_1_3_3_53_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0378-3758(02)00303-8"},{"key":"e_1_3_3_54_1","doi-asserted-by":"publisher","DOI":"10.1111\/j.1745-3984.2000.tb01076.x"},{"key":"e_1_3_3_55_1","doi-asserted-by":"publisher","DOI":"10.1177\/0013164402251034"},{"key":"e_1_3_3_56_1","doi-asserted-by":"publisher","DOI":"10.1177\/0013164410393238"},{"key":"e_1_3_3_57_1","doi-asserted-by":"publisher","DOI":"10.1111\/jedm.12263"},{"key":"e_1_3_3_58_1","doi-asserted-by":"publisher","DOI":"10.1177\/0146621604264870"},{"issue":"10","key":"e_1_3_3_59_1","first-page":"1","article-title":"Assessment of fit of item response theory models used in large-scale educational survey assessments","volume":"4","author":"van Rijn P. W.","year":"2016","unstructured":"van Rijn P. W., Sinharay S., Haberman S. J., Johnson M. S. (2016). Assessment of fit of item response theory models used in large-scale educational survey assessments. Large-scale Assessments in Education, 4(10), 1\u201323.","journal-title":"Large-scale Assessments in Education"},{"key":"e_1_3_3_60_1","doi-asserted-by":"publisher","DOI":"10.1177\/00131644221105819"},{"key":"e_1_3_3_61_1","doi-asserted-by":"publisher","DOI":"10.1177\/014662168100500212"},{"key":"e_1_3_3_62_1","doi-asserted-by":"publisher","DOI":"10.1002\/1097-0142(1950)3:1<32::AID-CNCR2820030106>3.0.CO;2-3"},{"key":"e_1_3_3_63_1","doi-asserted-by":"publisher","DOI":"10.1111\/bjep.12742"}],"container-title":["Educational and Psychological Measurement"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/00131644251369532","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/full-xml\/10.1177\/00131644251369532","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/00131644251369532","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,4,28]],"date-time":"2026-04-28T21:42:10Z","timestamp":1777412530000},"score":1,"resource":{"primary":{"URL":"https:\/\/journals.sagepub.com\/doi\/10.1177\/00131644251369532"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,10,4]]},"references-count":62,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2026,2]]}},"alternative-id":["10.1177\/00131644251369532"],"URL":"https:\/\/doi.org\/10.1177\/00131644251369532","relation":{},"ISSN":["0013-1644","1552-3888"],"issn-type":[{"value":"0013-1644","type":"print"},{"value":"1552-3888","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,10,4]]}}}