{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,30]],"date-time":"2026-10-30T20:37:40Z","timestamp":1793392660081,"version":"4.5.0"},"reference-count":34,"publisher":"Oxford University Press (OUP)","issue":"7","license":[{"start":{"date-parts":[[2024,5,20]],"date-time":"2024-05-20T00:00:00Z","timestamp":1716163200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Innovative Medicines Initiative 2 Joint Undertaking","award":["806968"],"award-info":[{"award-number":["806968"]}]},{"name":"European Union\u2019s Horizon 2020"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2024,6,20]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:sec>\n                    <jats:title>Objective<\/jats:title>\n                    <jats:p>This study evaluates regularization variants in logistic regression (L1, L2, ElasticNet, Adaptive L1, Adaptive ElasticNet, Broken adaptive ridge [BAR], and Iterative hard thresholding [IHT]) for discrimination and calibration performance, focusing on both internal and external validation.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Materials and Methods<\/jats:title>\n                    <jats:p>We use data from 5 US claims and electronic health record databases and develop models for various outcomes in a major depressive disorder patient population. We externally validate all models in the other databases. We use a train-test split of 75%\/25% and evaluate performance with discrimination and calibration. Statistical analysis for difference in performance uses Friedman\u2019s test and critical difference diagrams.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>Of the 840 models we develop, L1 and ElasticNet emerge as superior in both internal and external discrimination, with a notable AUC difference. BAR and IHT show the best internal calibration, without a clear external calibration leader. ElasticNet typically has larger model sizes than L1. Methods like IHT and BAR, while slightly less discriminative, significantly reduce model complexity.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusion<\/jats:title>\n                    <jats:p>L1 and ElasticNet offer the best discriminative performance in logistic regression for healthcare predictions, maintaining robustness across validations. For simpler, more interpretable models, L0-based methods (IHT and BAR) are advantageous, providing greater parsimony and calibration with fewer features. This study aids in selecting suitable regularization techniques for healthcare prediction models, balancing performance, complexity, and interpretability.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.1093\/jamia\/ocae109","type":"journal-article","created":{"date-parts":[[2024,5,9]],"date-time":"2024-05-09T08:44:03Z","timestamp":1715244243000},"page":"1514-1521","source":"Crossref","is-referenced-by-count":22,"title":["Comparing penalization methods for linear models on large observational health data"],"prefix":"10.1093","volume":"31","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-9085-9369","authenticated-orcid":false,"given":"Egill A","family":"Fridgeirsson","sequence":"first","affiliation":[{"name":"Department of Medical Informatics, Erasmus University Medical Center , 3015 GD Rotterdam, The Netherlands"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7723-417X","authenticated-orcid":false,"given":"Ross","family":"Williams","sequence":"additional","affiliation":[{"name":"Department of Medical Informatics, Erasmus University Medical Center , 3015 GD Rotterdam, The Netherlands"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Peter","family":"Rijnbeek","sequence":"additional","affiliation":[{"name":"Department of Medical Informatics, Erasmus University Medical Center , 3015 GD Rotterdam, The Netherlands"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9818-479X","authenticated-orcid":false,"given":"Marc A","family":"Suchard","sequence":"additional","affiliation":[{"name":"Department of Biostatistics, University of California, Los Angeles , Los Angeles, CA 90095-1772, United States"},{"name":"VA Informatics and Computing Infrastructure, United States Department of Veterans Affairs , Salt Lake City, UT 84148, United States"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2970-0778","authenticated-orcid":false,"given":"Jenna M","family":"Reps","sequence":"additional","affiliation":[{"name":"Department of Medical Informatics, Erasmus University Medical Center , 3015 GD Rotterdam, The Netherlands"},{"name":"Observational Health Data Analytics, Janssen Research and Development , Titusville, NJ 08560, United States"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"286","published-online":{"date-parts":[[2024,5,20]]},"reference":[{"issue":"5","key":"2024062007510064800_ocae109-B1","doi-asserted-by":"crossref","first-page":"983","DOI":"10.1093\/jamia\/ocac002","article-title":"Trends in the conduct and reporting of clinical prediction model development and validation: a systematic review","volume":"29","author":"Yang","year":"2022","journal-title":"J Am Med Inform Assoc"},{"issue":"1","key":"2024062007510064800_ocae109-B2","doi-asserted-by":"crossref","first-page":"267","DOI":"10.1111\/j.2517-6161.1996.tb02080.x","article-title":"Regression shrinkage and selection via the LASSO","volume":"58","author":"Tibshirani","year":"1996","journal-title":"J R Stat Soc B"},{"key":"2024062007510064800_ocae109-B3","doi-asserted-by":"crossref","first-page":"106394","DOI":"10.1016\/j.cmpb.2021.106394","article-title":"A standardized analytics pipeline for reliable and rapid development and validation of prediction models using observational health data","volume":"211","author":"Khalid","year":"2021","journal-title":"Comput Methods Programs Biomed"},{"issue":"1","key":"2024062007510064800_ocae109-B4","doi-asserted-by":"crossref","first-page":"25","DOI":"10.1016\/j.jclinepi.2014.09.007","article-title":"External validation of new risk prediction models is infrequent and reveals worse prognostic discrimination","volume":"68","author":"Siontis","year":"2015","journal-title":"J Clin Epidemiol"},{"issue":"1","key":"2024062007510064800_ocae109-B5","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/2414416.2414791","article-title":"Massive parallelization of serial inference algorithms for a complex generalized linear model","volume":"23","author":"Suchard","year":"2013","journal-title":"ACM Trans Model Comput Simul"},{"issue":"2","key":"2024062007510064800_ocae109-B6","doi-asserted-by":"crossref","first-page":"301","DOI":"10.1111\/j.1467-9868.2005.00503.x","article-title":"Regularization and variable selection via the elastic net","volume":"67","author":"Zou","year":"2005","journal-title":"J R Stat Soc B"},{"key":"2024062007510064800_ocae109-B7","first-page":"828","author":"Markus","year":"2022"},{"issue":"5","key":"2024062007510064800_ocae109-B8","doi-asserted-by":"crossref","first-page":"2133","DOI":"10.1214\/16-AOS1521","article-title":"False discoveries occur early on the LASSO path","volume":"45","author":"Su","year":"2017","journal-title":"Ann Stat"},{"key":"2024062007510064800_ocae109-B9","first-page":"53(3):1504","article-title":"Numerical characterization of support recovery in sparse regression with correlated design","author":"Kumar","year":"2024","journal-title":"Commun Stat Simul Comput"},{"issue":"1","key":"2024062007510064800_ocae109-B10","doi-asserted-by":"crossref","first-page":"55","DOI":"10.1080\/00401706.1970.10488634","article-title":"Ridge regression: biased estimation for nonorthogonal problems","volume":"12","author":"Hoerl","year":"1970","journal-title":"Technometrics"},{"issue":"174","key":"2024062007510064800_ocae109-B11","first-page":"1","article-title":"On the stability of feature selection algorithms","volume":"18","author":"Nogueira","year":"2018","journal-title":"J Mach Learn Res"},{"issue":"476","key":"2024062007510064800_ocae109-B12","doi-asserted-by":"crossref","first-page":"1418","DOI":"10.1198\/016214506000000735","article-title":"The adaptive LASSO and its oracle properties","volume":"101","author":"Zou","year":"2006","journal-title":"J Am Stat Assoc"},{"issue":"1","key":"2024062007510064800_ocae109-B13","doi-asserted-by":"crossref","first-page":"118","DOI":"10.1111\/insr.12469","article-title":"A critical review of LASSO and its derivatives for variable selection under dependence among covariates","volume":"90","author":"Freijeiro-Gonz\u00e1lez","year":"2022","journal-title":"Int Stat Rev"},{"key":"2024062007510064800_ocae109-B14","doi-asserted-by":"crossref","first-page":"104457","DOI":"10.1016\/j.chemolab.2021.104457","article-title":"A generalized stability estimator based on inter-intrastability of subsets for high-dimensional feature selection","volume":"220","author":"Wahid","year":"2022","journal-title":"Chemom Intell Lab Syst"},{"key":"2024062007510064800_ocae109-B15","doi-asserted-by":"crossref","first-page":"334","DOI":"10.1016\/j.jmva.2018.08.007","article-title":"Broken adaptive ridge regression and its asymptotic properties","volume":"168","author":"Dai","year":"2018","journal-title":"J Multivar Anal"},{"key":"2024062007510064800_ocae109-B16","doi-asserted-by":"crossref","first-page":"262","DOI":"10.1016\/j.jspi.2020.12.001","article-title":"A scalable surrogate L0 sparse regression method for generalized linear models with applications to large scale data","volume":"213","author":"Li","year":"2021","journal-title":"J Stat Plan Inference"},{"issue":"3","key":"2024062007510064800_ocae109-B17","doi-asserted-by":"crossref","first-page":"265","DOI":"10.1016\/j.acha.2009.04.002","article-title":"Iterative hard thresholding for compressed sensing","volume":"27","author":"Blumensath","year":"2009","journal-title":"Appl Comput Harmon Anal"},{"key":"2024062007510064800_ocae109-B18","doi-asserted-by":"crossref","first-page":"104762","DOI":"10.1016\/j.ijmedinf.2022.104762","article-title":"Logistic regression models for patient-level prediction based on massive observational data: do we need all data?","volume":"163","author":"John","year":"2022","journal-title":"Int J Med Inform"},{"issue":"1","key":"2024062007510064800_ocae109-B19","doi-asserted-by":"crossref","first-page":"311","DOI":"10.1186\/s12874-022-01793-5","article-title":"External validation of existing dementia prediction models on observational health data","volume":"22","author":"John","year":"2022","journal-title":"BMC Med Res Methodol"},{"issue":"8","key":"2024062007510064800_ocae109-B20","doi-asserted-by":"crossref","first-page":"969","DOI":"10.1093\/jamia\/ocy032","article-title":"Design and implementation of a standardized framework to generate and evaluate patient-level prediction models using observational healthcare data","volume":"25","author":"Reps","year":"2018","journal-title":"J Am Med Inform Assoc"},{"issue":"12","key":"2024062007510064800_ocae109-B21","doi-asserted-by":"crossref","first-page":"e050146","DOI":"10.1136\/bmjopen-2021-050146","article-title":"Investigating the impact of development and internal validation design when training prognostic models using a retrospective cohort in big US observational healthcare data","volume":"11","author":"Reps","year":"2021","journal-title":"BMJ Open"},{"issue":"1","key":"2024062007510064800_ocae109-B22","doi-asserted-by":"crossref","first-page":"W1","DOI":"10.7326\/M14-0698","article-title":"Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): explanation and elaboration","volume":"162","author":"Moons","year":"2015","journal-title":"Ann Intern Med"},{"issue":"3","key":"2024062007510064800_ocae109-B23","doi-asserted-by":"crossref","first-page":"e0118432","DOI":"10.1371\/journal.pone.0118432","article-title":"The precision-recall plot is more informative than the ROC plot when evaluating binary classifiers on imbalanced datasets","volume":"10","author":"Saito","year":"2015","journal-title":"PLoS One"},{"issue":"21","key":"2024062007510064800_ocae109-B24","doi-asserted-by":"crossref","first-page":"4051","DOI":"10.1002\/sim.8281","article-title":"The integrated calibration index (ICI) and related metrics for quantifying the calibration of logistic regression models","volume":"38","author":"Austin","year":"2019","journal-title":"Stat Med"},{"key":"2024062007510064800_ocae109-B25","first-page":"1","article-title":"Statistical comparisons of classifiers over multiple data sets","volume":"7","author":"Demsar","year":"2006","journal-title":"J Mach Learn Res"},{"issue":"4","key":"2024062007510064800_ocae109-B26","first-page":"579","article-title":"Best subset, forward stepwise or LASSO? Analysis and recommendations based on extensive comparisons","volume":"35","author":"Hastie","year":"2020","journal-title":"Stat Sci"},{"key":"2024062007510064800_ocae109-B27","doi-asserted-by":"crossref","first-page":"16","DOI":"10.1186\/s41512-019-0060-y","article-title":"Methodological standards for the development and evaluation of clinical prediction rules: a review of the literature","volume":"3","author":"Cowley","year":"2019","journal-title":"Diagn Progn Res"},{"issue":"6","key":"2024062007510064800_ocae109-B28","doi-asserted-by":"crossref","first-page":"565","DOI":"10.1177\/0272989X06295361","article-title":"Decision curve analysis: a novel method for evaluating prediction models","volume":"26","author":"Vickers","year":"2006","journal-title":"Med Decis Making"},{"issue":"456","key":"2024062007510064800_ocae109-B29","doi-asserted-by":"crossref","first-page":"1348","DOI":"10.1198\/016214501753382273","article-title":"Variable selection via nonconcave penalized likelihood and its oracle properties","volume":"96","author":"Fan","year":"2001","journal-title":"J Am Stat Assoc"},{"issue":"3","key":"2024062007510064800_ocae109-B30","doi-asserted-by":"crossref","first-page":"1103","DOI":"10.1214\/15-AOAS842","article-title":"Slope\u2014adaptive variable selection via convex optimization","volume":"9","author":"Bogdan","year":"2015","journal-title":"Ann Appl Stat"},{"issue":"2","key":"2024062007510064800_ocae109-B31","doi-asserted-by":"crossref","first-page":"98","DOI":"10.7326\/0003-4819-135-2-200107170-00010","article-title":"Excluding pulmonary embolism at the bedside without diagnostic imaging: management of patients with suspected pulmonary embolism presenting to the emergency department by using a simple clinical model and d-dimer","volume":"135","author":"Wells","year":"2001","journal-title":"Ann Intern Med"},{"issue":"2","key":"2024062007510064800_ocae109-B32","doi-asserted-by":"crossref","first-page":"263","DOI":"10.1378\/chest.09-1584","article-title":"Refining clinical risk stratification for predicting stroke and thromboembolism in atrial fibrillation using a novel risk factor-based approach: the Euro heart survey on atrial fibrillation","volume":"137","author":"Lip","year":"2010","journal-title":"Chest"},{"issue":"5","key":"2024062007510064800_ocae109-B33","doi-asserted-by":"crossref","first-page":"1093","DOI":"10.1378\/chest.10-0134","article-title":"A novel user-friendly score (HAS-BLED) to assess 1-year risk of major bleeding in patients with atrial fibrillation: the Euro Heart Survey","volume":"138","author":"Pisters","year":"2010","journal-title":"Chest"},{"issue":"5","key":"2024062007510064800_ocae109-B34","doi-asserted-by":"crossref","first-page":"373","DOI":"10.1016\/0021-9681(87)90171-8","article-title":"A new method of classifying prognostic comorbidity in longitudinal studies: development and validation","volume":"40","author":"Charlson","year":"1987","journal-title":"J Chronic Dis"}],"container-title":["Journal of the American Medical Informatics Association"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/jamia\/article-pdf\/31\/7\/1514\/58243644\/ocae109.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/jamia\/article-pdf\/31\/7\/1514\/58243644\/ocae109.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,6,20]],"date-time":"2024-06-20T03:52:45Z","timestamp":1718855565000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/jamia\/article\/31\/7\/1514\/7676584"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,5,20]]},"references-count":34,"journal-issue":{"issue":"7","published-online":{"date-parts":[[2024,5,20]]},"published-print":{"date-parts":[[2024,6,20]]}},"URL":"https:\/\/doi.org\/10.1093\/jamia\/ocae109","relation":{},"ISSN":["1067-5027","1527-974X"],"issn-type":[{"value":"1067-5027","type":"print"},{"value":"1527-974X","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2024,7]]},"published":{"date-parts":[[2024,5,20]]}}}