{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,19]],"date-time":"2026-01-19T05:28:51Z","timestamp":1768800531016,"version":"3.49.0"},"reference-count":53,"publisher":"MDPI AG","issue":"22","license":[{"start":{"date-parts":[[2021,11,16]],"date-time":"2021-11-16T00:00:00Z","timestamp":1637020800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100002347","name":"Federal Ministry of Education and Research","doi-asserted-by":"publisher","award":["FKZ: 01EC1902B"],"award-info":[{"award-number":["FKZ: 01EC1902B"]}],"id":[{"id":"10.13039\/501100002347","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["IJERPH"],"abstract":"<jats:p>(1) Background: Predicting chronic low back pain (LBP) is of clinical and economic interest as LBP leads to disabilities and health service utilization. This study aims to build a competitive and interpretable prediction model; (2) Methods: We used clinical and claims data of 3837 participants of a population-based cohort study to predict future LBP consultations (ICD-10: M40.XX-M54.XX). Best subset selection (BSS) was applied in repeated random samples of training data (75% of data); scoring rules were used to identify the best subset of predictors. The rediction accuracy of BSS was compared to randomforest and support vector machines (SVM) in the validation data (25% of data); (3) Results: The best subset comprised 16 out of 32 predictors. Previous occurrence of LBP increased the odds for future LBP consultations (odds ratio (OR) 6.91 [5.05; 9.45]), while concomitant diseases reduced the odds (1 vs. 0, OR: 0.74 [0.57; 0.98], &gt;1 vs. 0: 0.37 [0.21; 0.67]). The area-under-curve (AUC) of BSS was acceptable (0.78 [0.74; 0.82]) and comparable with SVM (0.78 [0.74; 0.82]) and randomforest (0.79 [0.75; 0.83]); (4) Conclusions: Regarding prediction accuracy, BSS has been considered competitive with established machine-learning approaches. Nonetheless, considerable misclassification is inherent and further refinements are required to improve predictions.<\/jats:p>","DOI":"10.3390\/ijerph182212013","type":"journal-article","created":{"date-parts":[[2021,11,16]],"date-time":"2021-11-16T11:32:03Z","timestamp":1637062323000},"page":"12013","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":4,"title":["Predicting Physician Consultations for Low Back Pain Using Claims Data and Population-Based Cohort Data\u2014An Interpretable Machine Learning Approach"],"prefix":"10.3390","volume":"18","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-3372-2021","authenticated-orcid":false,"given":"Adrian","family":"Richter","sequence":"first","affiliation":[{"name":"Department SHIP-KEF, Institute for Community Medicine, Greifswald University Medical Center, Walther Rathenau Str. 48, 17475 Greifswald, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Julia","family":"Truthmann","sequence":"additional","affiliation":[{"name":"Department of Family Medicine, Institute for Community Medicine, Fleischmannstr. 42, 17475 Greifswald, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8877-2950","authenticated-orcid":false,"given":"Jean-Fran\u00e7ois","family":"Chenot","sequence":"additional","affiliation":[{"name":"Department of Family Medicine, Institute for Community Medicine, Fleischmannstr. 42, 17475 Greifswald, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5266-9396","authenticated-orcid":false,"given":"Carsten Oliver","family":"Schmidt","sequence":"additional","affiliation":[{"name":"Department SHIP-KEF, Institute for Community Medicine, Greifswald University Medical Center, Walther Rathenau Str. 48, 17475 Greifswald, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2021,11,16]]},"reference":[{"key":"ref_1","first-page":"883","article-title":"Non-Specific Low Back Pain","volume":"114","author":"Chenot","year":"2017","journal-title":"Dtsch. Aerzteblatt Online"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"736","DOI":"10.1016\/S0140-6736(16)30970-9","article-title":"Non-specific low back pain","volume":"389","author":"Maher","year":"2017","journal-title":"Lancet"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"280","DOI":"10.1016\/j.ejpain.2008.04.005","article-title":"Costs of back pain in Germany","volume":"13","author":"Wenig","year":"2009","journal-title":"Eur. J. Pain"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"323","DOI":"10.1136\/bmj.327.7410.323","article-title":"Acute low back pain: Systematic review of its prognosis","volume":"327","author":"Pengel","year":"2003","journal-title":"BMJ"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"149","DOI":"10.1007\/s00586-002-0508-5","article-title":"Low back pain: What is the long-term course? A review of studies of general patient populations","volume":"12","author":"Hestbaek","year":"2003","journal-title":"Eur. Spine J."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"1660","DOI":"10.1002\/acr.23811","article-title":"Course of Back Pain in the Canadian Population: Trajectories, Predictors, and Outcomes","volume":"71","author":"Canizares","year":"2019","journal-title":"Arthritis Rheum."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"5","DOI":"10.1023\/A:1010933404324","article-title":"Random forests","volume":"45","author":"Breiman","year":"2001","journal-title":"Mach. Learn."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Weng, S.F., Reps, J.M., Kai, J., Garibaldi, J.M., and Qureshi, N. (2017). Can machine-learning improve cardiovascular risk prediction using routine clinical data?. PLoS ONE, 12.","DOI":"10.1371\/journal.pone.0174944"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"564","DOI":"10.1002\/bimj.201300077","article-title":"Probability estimation with machine learning methods for dichotomous and multicategory outcome: Applications","volume":"56","author":"Kruppa","year":"2014","journal-title":"Biom. J."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"588","DOI":"10.1002\/bimj.201300226","article-title":"Machine learning versus statistical modeling","volume":"56","author":"Boulesteix","year":"2014","journal-title":"Biom. J."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"357","DOI":"10.1093\/biomet\/54.3-4.357","article-title":"The discarding of variables in multivariate analysis","volume":"54","author":"Beale","year":"1967","journal-title":"Biometrika"},{"key":"ref_12","first-page":"579","article-title":"Best Subset, Forward Stepwise or Lasso? Analysis and Recommendations Based on Extensive Comparisons","volume":"35","author":"Hastie","year":"2020","journal-title":"Stat. Sci."},{"key":"ref_13","first-page":"294","article-title":"Cohort Profile: The Study of Health in Pomerania","volume":"40","author":"Alte","year":"2010","journal-title":"Int. J. Epidemiol."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"606","DOI":"10.1046\/j.1525-1497.2001.016009606.x","article-title":"The PHQ-9: Validity of a brief depression severity measure","volume":"16","author":"Kroenke","year":"2001","journal-title":"J. Gen. Intern. Med."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"133","DOI":"10.1016\/0304-3959(92)90154-4","article-title":"Grading the severity of chronic pain","volume":"50","author":"Ormel","year":"1992","journal-title":"Pain"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"2005","DOI":"10.1097\/BRS.0b013e318133fad8","article-title":"Back Pain in the German Adult Population","volume":"32","author":"Schmidt","year":"2007","journal-title":"Spine"},{"key":"ref_17","unstructured":"Das Bundesgesundheitsministerium (2021, November 15). Das deutsche Gesundheitssystem\u2014Leistungsstark. Sicher. Bew\u00e4hrt. Available online: https:\/\/www.bundesgesundheitsministerium.de\/fileadmin\/Dateien\/5_Publikationen\/Gesundheit\/Broschueren\/200629_BMG_Das_deutsche_Gesundheitssystem_DE.pdf."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"946","DOI":"10.1016\/j.is.2012.11.005","article-title":"A taxonomy of privacy-preserving record linkage techniques","volume":"38","author":"Vatsalan","year":"2013","journal-title":"Inf. Syst."},{"key":"ref_19","unstructured":"R Development Core Team (2020). R: A Language and Environment for Statistical Computing, R Foundation for Statistical Computing."},{"key":"ref_20","unstructured":"Jackman, S. (2021, August 08). pscl: Classes and Methods for R Developed in the Political Science Computational Laboratory. Available online: http:\/\/github.com\/atahk\/pscl."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"1","DOI":"10.18637\/jss.v027.i08","article-title":"Regression Models for Count Data inR","volume":"27","author":"Zeileis","year":"2008","journal-title":"J. Stat. Softw."},{"key":"ref_22","unstructured":"Weston, S., and Microsoft Corporation (2021, August 08). doParallel: Foreach Parallel Adaptor for the \u2018parallel\u2019 Package. Available online: https:\/\/CRAN.R-project.org\/package=doParallel."},{"key":"ref_23","unstructured":"University of Greifswald (2021, August 08). HPC Brain Cluster. Available online: https:\/\/rz.uni-greifswald.de\/dienste\/allgemein\/sonstiges\/high-performance-computing\/."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"738","DOI":"10.2307\/2532959","article-title":"A Score Test for Zero Inflation in a Poisson Distribution","volume":"51","author":"Broek","year":"1995","journal-title":"Biometrics"},{"key":"ref_25","unstructured":"Friendly, M. (2021, August 08). vcdExtra: \u2019vcd\u2019 Extensions and Additions. Available online: https:\/\/CRAN.R-project.org\/package=vcdExtra."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"1288","DOI":"10.1016\/j.jclinepi.2004.03.012","article-title":"New ICD-10 version of the Charlson comorbidity index predicted in-hospital mortality","volume":"57","author":"Sundararajan","year":"2004","journal-title":"J. Clin. Epidemiol."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Hofner, B., Boccuto, L., and G\u00f6ker, M. (2015). Controlling false discoveries in high-dimensional situations: Boosting with stability selection. BMC Bioinform., 16.","DOI":"10.1186\/s12859-015-0575-3"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"6083072","DOI":"10.1155\/2017\/6083072","article-title":"An Update on Statistical Boosting in Biomedicine","volume":"2017","author":"Mayr","year":"2017","journal-title":"Comput. Math. Methods Med."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"160","DOI":"10.1002\/cem.1225","article-title":"Repeated double cross validation","volume":"23","author":"Filzmoser","year":"2009","journal-title":"J. Chemom."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"261","DOI":"10.1177\/0049124104268644","article-title":"Multimodel Inference: Understanding AIC and BIC in Model Selection","volume":"33","author":"Burnham","year":"2004","journal-title":"Sociol. Methods Res."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"359","DOI":"10.1198\/016214506000001437","article-title":"Strictly Proper Scoring Rules, Prediction, and Estimation","volume":"102","author":"Gneiting","year":"2007","journal-title":"J. Am. Stat. Assoc."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"296","DOI":"10.1080\/00031305.2016.1173590","article-title":"Visualizing Count Data Regressions Using Rootograms","volume":"70","author":"Kleiber","year":"2016","journal-title":"Am. Stat."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"1","DOI":"10.18637\/jss.v079.c02","article-title":"plotROC: A Tool for Plotting ROC Curves","volume":"79","author":"Sachs","year":"2017","journal-title":"J. Stat. Softw."},{"key":"ref_34","first-page":"1","article-title":"Mice: Multivariate Imputation by Chained Equations in R","volume":"45","year":"2011","journal-title":"J. Stat. Softw."},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Jakobsen, J.C., Gluud, C., Wetterslev, J., and Winkel, P. (2017). When and how should multiple imputation be used for handling missing data in randomised clinical trials\u2014A practical guide with flowcharts. BMC Med. Res. Methodol., 17.","DOI":"10.1186\/s12874-017-0442-1"},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"63","DOI":"10.1016\/j.jclinepi.2019.02.016","article-title":"The proportion of missing data should not be used to guide decisions on multiple imputation","volume":"110","author":"Hughes","year":"2019","journal-title":"J. Clin. Epidemiol."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"273","DOI":"10.1007\/BF00994018","article-title":"Support vector machines","volume":"20","author":"Vapnik","year":"1995","journal-title":"Mach. Learn."},{"key":"ref_38","unstructured":"Bischl, B., Binder, M., Lang, M., Pielok, T., Richter, J., Coors, S., Thomas, J., Ullmann, T., Becker, M., and Boulesteix, A.-L. (2021). Hyperparameter Optimization: Foundations, Algorithms, Best Practices and Open Challenges. arXiv."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"1301","DOI":"10.1002\/widm.1301","article-title":"Hyperparameters and tuning strategies for random forest","volume":"9","author":"Probst","year":"2019","journal-title":"Wiley Interdiscip. Rev. Data Min. Knowl. Discov."},{"key":"ref_40","unstructured":"Hsu, C.-W., Chang, C.-C., and Lin, C.-J. (2021, August 08). A Practical Guide to Support Vector Classification (Update 2016). Available online: https:\/\/www.csie.ntu.edu.tw\/~cjlin\/papers\/guide\/guide.pdf."},{"key":"ref_41","unstructured":"Meyer, D., Dimitriadou, E., Hornik, K., Weingessel, A., and Leisch, F. (2021, August 08). Misc Functions of the Department of Statistics, Probability Theory Group (Formerly: E1071). Available online: http:\/\/packages.renjin.org\/package\/org.renjin.cran\/e1071."},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Hosmer, D.W., Lemeshow, S., and Sturdivant, R.X. (2013). Applied Logistic Regression, John Wiley & Sons.","DOI":"10.1002\/9781118548387"},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"12","DOI":"10.1016\/j.jclinepi.2019.02.004","article-title":"A systematic review shows no performance benefit of machine learning over logistic regression for clinical prediction models","volume":"110","author":"Christodoulou","year":"2019","journal-title":"J. Clin. Epidemiol."},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Karran, E.L., McAuley, J.H., Traeger, A.C., Hillier, S.L., Grabherr, L., Russek, L.N., and Moseley, G.L. (2017). Can screening instruments accurately determine poor outcome risk in adults with recent onset low back pain? A systematic review and meta-analysis. BMC Med., 15.","DOI":"10.1186\/s12916-016-0774-4"},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"748","DOI":"10.1097\/AJP.0000000000000591","article-title":"Lack of Prognostic Model Validation in Low Back Pain Prediction Studies","volume":"34","author":"McIntosh","year":"2018","journal-title":"Clin. J. Pain"},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"275","DOI":"10.1016\/j.ejpain.2007.06.004","article-title":"The impact of specialist care for low back pain on health service utilization in primary care patients: A prospective cohort study","volume":"12","author":"Chenot","year":"2008","journal-title":"Eur. J. Pain"},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"747.e1","DOI":"10.1016\/j.ejpain.2009.11.005","article-title":"Factors defining care-seeking in low back pain\u2014A meta-analysis of population based surveys","volume":"14","author":"Ferreira","year":"2010","journal-title":"Eur. J. Pain"},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"3762651","DOI":"10.1155\/2017\/3762651","article-title":"Defining an Optimal Cut-Point Value in ROC Analysis: An Alternative Approach","volume":"2017","author":"Unal","year":"2017","journal-title":"Comput. Math. Methods Med."},{"key":"ref_49","doi-asserted-by":"crossref","first-page":"153","DOI":"10.3344\/kjp.2020.33.2.153","article-title":"A prediction model of low back pain risk: A population based cohort study in Korea","volume":"33","author":"Mukasa","year":"2020","journal-title":"Korean J. Pain"},{"key":"ref_50","doi-asserted-by":"crossref","first-page":"12","DOI":"10.1093\/fampra\/cmq072","article-title":"Psychosocial risk factors for chronic low back pain in primary care--a systematic review","volume":"28","author":"Ramond","year":"2010","journal-title":"Fam. Pract."},{"key":"ref_51","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.ijmedinf.2017.09.013","article-title":"Prediction of lung cancer patient survival via supervised machine learning classification techniques","volume":"108","author":"Lynch","year":"2017","journal-title":"Int. J. Med. Inform."},{"key":"ref_52","unstructured":"Paluszynska, A., Biecek, P., Jiang, Y., and Jiang, M.Y. (2021, August 08). Package \u2018randomForestExplainer\u2019. Available online: http:\/\/cran.nexr.com\/web\/packages\/randomForestExplainer\/randomForestExplainer.pdf."},{"key":"ref_53","doi-asserted-by":"crossref","first-page":"813","DOI":"10.1214\/15-AOS1388","article-title":"Best subset selection via a modern optimization lens","volume":"44","author":"Bertsimas","year":"2016","journal-title":"Ann. Stat."}],"container-title":["International Journal of Environmental Research and Public Health"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1660-4601\/18\/22\/12013\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T07:31:04Z","timestamp":1760167864000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1660-4601\/18\/22\/12013"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,11,16]]},"references-count":53,"journal-issue":{"issue":"22","published-online":{"date-parts":[[2021,11]]}},"alternative-id":["ijerph182212013"],"URL":"https:\/\/doi.org\/10.3390\/ijerph182212013","relation":{},"ISSN":["1660-4601"],"issn-type":[{"value":"1660-4601","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,11,16]]}}}