{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T08:28:33Z","timestamp":1760171313401,"version":"build-2065373602"},"reference-count":58,"publisher":"MDPI AG","issue":"9","license":[{"start":{"date-parts":[[2023,9,18]],"date-time":"2023-09-18T00:00:00Z","timestamp":1694995200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Algorithms"],"abstract":"<jats:p>This article reviews several implementation aspects in estimating regularized single-group and multiple-group structural equation models (SEM). It is demonstrated that approximate estimation approaches that rely on a differentiable approximation of non-differentiable penalty functions perform similarly to the coordinate descent optimization approach of regularized SEMs. Furthermore, using a fixed regularization parameter can sometimes be superior to an optimal regularization parameter selected by the Bayesian information criterion when it comes to the estimation of structural parameters. Moreover, the widespread penalty functions of regularized SEM implemented in several R packages were compared with the estimation based on a recently proposed penalty function in the Mplus software. Finally, we also investigate the performance of a clever replacement of the optimization function in regularized SEM with a smoothed differentiable approximation of the Bayesian information criterion proposed by O\u2019Neill and Burke in 2023. The findings were derived through two simulation studies and are intended to guide the practical implementation of regularized SEM in future software pieces.<\/jats:p>","DOI":"10.3390\/a16090446","type":"journal-article","created":{"date-parts":[[2023,9,19]],"date-time":"2023-09-19T02:47:52Z","timestamp":1695091672000},"page":"446","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":6,"title":["Implementation Aspects in Regularized Structural Equation Models"],"prefix":"10.3390","volume":"16","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-8226-3132","authenticated-orcid":false,"given":"Alexander","family":"Robitzsch","sequence":"first","affiliation":[{"name":"IPN\u2013Leibniz Institute for Science and Mathematics Education, Olshausenstra\u00dfe 62, 24118 Kiel, Germany"},{"name":"Centre for International Student Assessment (ZIB), Olshausenstra\u00dfe 62, 24118 Kiel, Germany"}]}],"member":"1968","published-online":{"date-parts":[[2023,9,18]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Bartholomew, D.J., Knott, M., and Moustaki, I. (2011). Latent Variable Models and Factor Analysis: A Unified Approach, Wiley.","DOI":"10.1002\/9781119970583"},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Bollen, K.A. (1989). Structural Equations with Latent Variables, Wiley.","DOI":"10.1002\/9781118619179"},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Arminger, G., Clogg, C.C., and Sobel, M.E. (1995). Handbook of Statistical Modeling for the Social and Behavioral Sciences, Springer.","DOI":"10.1007\/978-1-4899-1292-3"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"J\u00f6reskog, K.G., Olsson, U.H., and Wallentin, F.Y. (2016). Multivariate Analysis with LISREL, Springer.","DOI":"10.1007\/978-3-319-33153-9"},{"key":"ref_5","unstructured":"Kaplan, D. (2009). Structural Equation Modeling: Foundations and Extensions, Sage."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Edwards, M.C., and MacCallum, R.C. (2012). Current Topics in the Theory and Application of Latent Variable Models, Routledge.","DOI":"10.4324\/9780203813409"},{"key":"ref_7","first-page":"297","article-title":"Structural equation modeling","volume":"Volume 26","author":"Rao","year":"2007","journal-title":"Handbook of Statistics"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Magnus, J.R., and Neudecker, H. (2019). Matrix Differential Calculus With Applications in Statistics and Econometrics, Wiley.","DOI":"10.1002\/9781119541219"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"523","DOI":"10.1080\/10705510903008261","article-title":"Two rules of identification for structural equation models","volume":"16","author":"Bollen","year":"2009","journal-title":"Struct. Equ. Model."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"865","DOI":"10.1214\/10-AOS859","article-title":"Global identifiability of linear structural equation models","volume":"39","author":"Drton","year":"2011","journal-title":"Ann. Stat."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"555","DOI":"10.1080\/10705511.2016.1154793","article-title":"Regularized structural equation modeling","volume":"23","author":"Jacobucci","year":"2016","journal-title":"Struct. Equ. Model."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Robitzsch, A. (2023). Model-robust estimation of multiple-group structural equation models. Algorithms, 16.","DOI":"10.31234\/osf.io\/25md9"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Hoyle, R.H. (2023). Handbook of Structural Equation Modeling, Guilford Press.","DOI":"10.1037\/0000319-021"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Orzek, J.H., Arnold, M., and Voelkle, M.C. (2023). Striving for sparsity: On exact and approximate solutions in regularized structural equation models. Struct. Equ. Model., Epub ahead of print.","DOI":"10.1080\/10705511.2023.2189070"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"579","DOI":"10.3390\/psych3040038","article-title":"Tutorial on the use of the regsem package in R","volume":"3","author":"Li","year":"2021","journal-title":"Psych"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Asparouhov, T., and Muth\u00e9n, B. (2023, March 28). Penalized Structural Equation Models. Technical Report. Available online: https:\/\/rb.gy\/tbaj7.","DOI":"10.1080\/10705511.2023.2263913"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"71","DOI":"10.1007\/s11222-023-10204-8","article-title":"Variable selection using a smooth information criterion for distributional regression models","volume":"33","author":"Burke","year":"2023","journal-title":"Stat. Comput."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Boos, D.D., and Stefanski, L.A. (2013). Essential Statistical Inference, Springer.","DOI":"10.1007\/978-1-4614-4818-1"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"119","DOI":"10.1111\/j.1467-9531.2011.01236.x","article-title":"Biases of parameter estimates in misspecified structural equation models","volume":"41","author":"Kolenikov","year":"2011","journal-title":"Sociol. Methodol."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"1","DOI":"10.2307\/1912526","article-title":"Maximum likelihood estimation of misspecified models","volume":"50","author":"White","year":"1982","journal-title":"Econometrica"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"329","DOI":"10.1007\/s11336-017-9566-9","article-title":"A penalized likelihood method for structural equation modeling","volume":"82","author":"Huang","year":"2017","journal-title":"Psychometrika"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"1013","DOI":"10.1109\/TNNLS.2012.2197412","article-title":"L1\/2 regularization: A thresholding representation theory and a fast solver","volume":"23","author":"Xu","year":"2012","journal-title":"IEEE Trans. Neur. Net. Lear."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Hastie, T., Tibshirani, R., and Wainwright, M. (2015). Statistical Learning with Sparsity: The Lasso and Generalizations, CRC Press.","DOI":"10.1201\/b18401"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"1348","DOI":"10.1198\/016214501753382273","article-title":"Variable selection via nonconcave penalized likelihood and its oracle properties","volume":"96","author":"Fan","year":"2001","journal-title":"J. Am. Stat. Assoc."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Fan, J., Li, R., Zhang, C.H., and Zou, H. (2020). Statistical Foundations of Data Science, Chapman and Hall\/CRC.","DOI":"10.1201\/9780429096280"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"5755","DOI":"10.1038\/s41598-020-62473-2","article-title":"Meta-analysis based on nonconvex regularization","volume":"10","author":"Zhang","year":"2020","journal-title":"Sci. Rep."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"499","DOI":"10.1111\/bmsp.12130","article-title":"A penalized likelihood method for multi-group structural equation modelling","volume":"71","author":"Huang","year":"2018","journal-title":"Br. Math. Stat. Psychol."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"660","DOI":"10.1007\/s11336-016-9545-6","article-title":"Regularized latent class analysis with application in cognitive diagnosis","volume":"82","author":"Chen","year":"2017","journal-title":"Psychometrika"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"312","DOI":"10.1198\/jasa.2009.tm08013","article-title":"Regularization parameter selections via generalized information criterion","volume":"105","author":"Zhang","year":"2010","journal-title":"J. Am. Stat. Assoc."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"800","DOI":"10.1080\/10705511.2022.2039660","article-title":"Partially confirmatory approach to factor analysis with Bayesian learning: A LAWBL tutorial","volume":"22","author":"Chen","year":"2022","journal-title":"Struct. Equ. Model."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"65","DOI":"10.1007\/s11336-021-09751-8","article-title":"Single- and multiple-group penalized factor analysis: A trust-region algorithm approach with integrated automatic multiple tuning parameter selection","volume":"86","author":"Geminiani","year":"2021","journal-title":"Psychometrika"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Hirose, K., and Terada, Y. (2022). Sparse and simple structure estimation via prenet penalization. Psychometrika, Epub ahead of print.","DOI":"10.1007\/s11336-022-09868-4"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"1","DOI":"10.18637\/jss.v093.i07","article-title":"lslx: Semi-confirmatory structural equation modeling via penalized likelihood","volume":"93","author":"Huang","year":"2020","journal-title":"J. Stat. Softw."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"576","DOI":"10.1080\/10705511.2018.1558060","article-title":"Should regularization replace simple structure rotation in exploratory factor analysis?","volume":"26","author":"Scharf","year":"2019","journal-title":"Struct. Equ. Model."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"811","DOI":"10.1080\/00273171.2019.1681252","article-title":"Regularized estimation of the nominal response model","volume":"55","author":"Battauz","year":"2020","journal-title":"Multivar. Behav. Res."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"97","DOI":"10.1007\/s11634-015-0205-y","article-title":"A uniform framework for the combination of penalties in generalized structured models","volume":"11","author":"Oelker","year":"2017","journal-title":"Adv. Data Anal. Classif."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"631","DOI":"10.3390\/stats5030039","article-title":"Comparing the robustness of the structural after measurement (SAM) approach to structural equation modeling (SEM) against local model misspecifications with alternative estimation approaches","volume":"5","author":"Robitzsch","year":"2022","journal-title":"Stats"},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"161","DOI":"10.1177\/1471082X16642560","article-title":"Regularized regression for categorical data","volume":"16","author":"Tutz","year":"2016","journal-title":"Stat. Model."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"389","DOI":"10.1177\/1471082X14553366","article-title":"Selection and fusion of categorical predictors with L0-type penalties","volume":"15","author":"Oelker","year":"2015","journal-title":"Stat. Model."},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Phan, D.T., and Id\u00e9, T. (2019, January 2\u20134). l0-regularized sparsity for probabilistic mixture models. Proceedings of the 2019 SIAM International Conference on Data Mining, Calgary, AB, Canada.","DOI":"10.1137\/1.9781611975673.20"},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"223","DOI":"10.1080\/01621459.2011.645783","article-title":"Likelihood-based selection and sharp parameter estimation","volume":"107","author":"Shen","year":"2012","journal-title":"J. Am. Stat. Assoc."},{"key":"ref_42","unstructured":"Lee, S.Y. (2007). Handbook of Latent Variable and Related Models, Elsevier."},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"495","DOI":"10.1080\/10705511.2014.919210","article-title":"Multiple-group factor analysis alignment","volume":"21","author":"Asparouhov","year":"2014","journal-title":"Struct. Equ. Model."},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Millsap, R.E. (2011). Statistical Approaches to Measurement Invariance, Routledge.","DOI":"10.4324\/9780203821961"},{"key":"ref_45","unstructured":"R Core Team (2023). R: A Language and Environment for Statistical Computing, R Core Team. Available online: https:\/\/www.R-project.org\/."},{"key":"ref_46","unstructured":"Robitzsch, A. (2023, August 11). sirt: Supplementary Item Response Theory Models; R package version 3.13-228. Available online: https:\/\/CRAN.R-project.org\/package=sirt."},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"24","DOI":"10.1111\/emip.12486","article-title":"The multidimensionality of measurement bias in high-stakes testing: Using machine learning to evaluate complex sources of differential item functioning","volume":"42","author":"Belzak","year":"2023","journal-title":"Educ. Meas."},{"key":"ref_48","doi-asserted-by":"crossref","unstructured":"Chen, Y., Li, C., Ouyang, J., and Xu, G. (2023). DIF statistical inference without knowing anchoring items. Psychometrika, Epub ahead of print.","DOI":"10.1007\/s11336-023-09930-9"},{"key":"ref_49","doi-asserted-by":"crossref","first-page":"192","DOI":"10.3390\/stats6010012","article-title":"Comparing robust linking and regularized estimation for linking two groups in the 1PL and 2PL models in the presence of sparse uniform differential item functioning","volume":"6","author":"Robitzsch","year":"2023","journal-title":"Stats"},{"key":"ref_50","doi-asserted-by":"crossref","first-page":"921","DOI":"10.1007\/s11336-016-9529-6","article-title":"Latent variable selection for multidimensional item response theory models via L1 regularization","volume":"81","author":"Sun","year":"2016","journal-title":"Psychometrika"},{"key":"ref_51","doi-asserted-by":"crossref","first-page":"21","DOI":"10.1007\/s11336-013-9377-6","article-title":"A penalty approach to differential item functioning in Rasch models","volume":"80","author":"Tutz","year":"2015","journal-title":"Psychometrika"},{"key":"ref_52","doi-asserted-by":"crossref","first-page":"1473","DOI":"10.1007\/s11336-022-09863-9","article-title":"Computation for latent variable model estimation: A unified stochastic proximal framework","volume":"87","author":"Zhang","year":"2022","journal-title":"Psychometrika"},{"key":"ref_53","doi-asserted-by":"crossref","first-page":"850","DOI":"10.1080\/01621459.2014.934827","article-title":"Statistical analysis of Q-matrix based diagnostic classification models","volume":"110","author":"Chen","year":"2015","journal-title":"J. Am. Stat. Assoc."},{"key":"ref_54","doi-asserted-by":"crossref","unstructured":"Robitzsch, A. (2020). Regularized latent class analysis for polytomous item responses: An application to SPM-LS data. J. Intell., 8.","DOI":"10.20944\/preprints202007.0269.v1"},{"key":"ref_55","doi-asserted-by":"crossref","first-page":"1284","DOI":"10.1080\/01621459.2017.1340889","article-title":"Identifying latent structures in restricted latent class models","volume":"113","author":"Xu","year":"2018","journal-title":"J. Am. Stat. Assoc."},{"key":"ref_56","doi-asserted-by":"crossref","unstructured":"Robitzsch, A. (2022). Regularized mixture Rasch model. Information, 13.","DOI":"10.3390\/info13110534"},{"key":"ref_57","unstructured":"Wallin, G., Chen, Y., and Moustaki, I. (2023). DIF analysis with unknown groups and anchor items. arXiv."},{"key":"ref_58","doi-asserted-by":"crossref","first-page":"108","DOI":"10.1006\/jmps.1999.1279","article-title":"Cross-validation methods","volume":"44","author":"Browne","year":"2000","journal-title":"J. Math. Psychol."}],"container-title":["Algorithms"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1999-4893\/16\/9\/446\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T20:53:03Z","timestamp":1760129583000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1999-4893\/16\/9\/446"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,9,18]]},"references-count":58,"journal-issue":{"issue":"9","published-online":{"date-parts":[[2023,9]]}},"alternative-id":["a16090446"],"URL":"https:\/\/doi.org\/10.3390\/a16090446","relation":{},"ISSN":["1999-4893"],"issn-type":[{"type":"electronic","value":"1999-4893"}],"subject":[],"published":{"date-parts":[[2023,9,18]]}}}