{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,28]],"date-time":"2026-09-28T15:52:29Z","timestamp":1790610749986,"version":"4.1.0"},"reference-count":35,"publisher":"Ovid Technologies (Wolters Kluwer Health)","issue":"6","content-domain":{"domain":["lww.com","ovid.com"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2021,6]]},"abstract":"<jats:p>Researchers often convert prediction tools built on statistical regression models into integer scores and risk classification systems in the name of simplicity. However, this workflow discards useful information and reduces prediction accuracy. We, therefore, investigated the impact on prediction accuracy when researchers simplify a regression model into an integer score using a simulation study and an example clinical data set. Simulated independent training and test sets (n = 1000) were randomly generated such that a logistic regression model would perform at a specified target area under the receiver operating characteristic curve (AUC) of 0.7, 0.8, or 0.9. After fitting a logistic regression with continuous covariates to each data set, continuous variables were dichotomized using data-dependent cut points. A logistic regression was refit, and the coefficients were scaled and rounded to create an integer score. A risk classification system was built by stratifying integer scores into low-, intermediate-, and high-risk tertiles. Discrimination and calibration were assessed by calculating the AUC and index of prediction accuracy (IPA) for each model. The optimism in performance between the training set and test set was calculated for both AUC and IPA. The logistic regression model using the continuous form of covariates outperformed all other models. In the simulation study, converting the logistic regression model to an integer score and subsequent risk classification system incurred an average decrease of 0.057\u20130.094 in AUC, and an absolute 6.2%\u201317.5% in IPA. The largest decrease in both AUC and IPA occurred in the dichotomization step. The dichotomization and risk stratification steps also increased the optimism of the resulting models, such that they appeared to be able to predict better than they actually would on new data. In the clinical data set, converting the logistic regression with continuous covariates to an integer score incurred a decrease in externally validated AUC of 0.06 and a decrease in externally validated IPA of 13%. Converting a regression model to an integer score decreases model performance considerably. Therefore, we recommend developing a regression model that incorporates all available information to make the most accurate predictions possible, and using the unaltered regression model when making predictions for individual patients. In all cases, researchers should be mindful that they correctly validate the specific model that is intended for clinical use.<\/jats:p>","DOI":"10.1213\/ane.0000000000005362","type":"journal-article","created":{"date-parts":[[2021,1,15]],"date-time":"2021-01-15T13:09:19Z","timestamp":1610716159000},"page":"1603-1613","update-policy":"https:\/\/doi.org\/10.1097\/lww.0000000000001000","source":"Crossref","is-referenced-by-count":23,"title":["Developing a Clinical Prediction Score: Comparing Prediction Accuracy of Integer Scores to Statistical Regression Models"],"prefix":"10.1213","volume":"132","author":[{"given":"Vigneshwar","family":"Subramanian","sequence":"first","affiliation":[{"name":"Cleveland Clinic Lerner College of Medicine at Case Western Reserve University, Cleveland, Ohio"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Edward J.","family":"Mascha","sequence":"additional","affiliation":[{"name":"Departments of Quantitative Health Sciences and Outcomes Research and"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Michael W.","family":"Kattan","sequence":"additional","affiliation":[{"name":"Department of Quantitative Health Sciences, Lerner Research Institute, Cleveland Clinic, Cleveland, Ohio."}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"276","published-online":{"date-parts":[[2021,1,15]]},"reference":[{"key":"R1-20250502","doi-asserted-by":"crossref","first-page":"1338","DOI":"10.1097\/ALN.0b013e3181fc6e0a","article-title":"Prediction of postoperative pulmonary complications in a population-based surgical cohort.","volume":"113","author":"Canet","year":"2010","journal-title":"Anesthesiology"},{"key":"R2-20250502","doi-asserted-by":"crossref","first-page":"1404","DOI":"10.1093\/eurheartj\/ehs337","article-title":"Predicting survival in heart failure: a risk score based on 39 372 patients from 30 studies.","volume":"34","author":"Pocock","year":"2013","journal-title":"Eur Heart J"},{"key":"R3-20250502","doi-asserted-by":"crossref","first-page":"2556","DOI":"10.1016\/j.jacc.2009.09.076","article-title":"A risk score to predict bleeding in patients with acute coronary syndromes.","volume":"55","author":"Mehran","year":"2010","journal-title":"J Am Coll Cardiol"},{"key":"R4-20250502","doi-asserted-by":"crossref","first-page":"1397","DOI":"10.1016\/j.jacc.2005.01.041","article-title":"Prediction of mortality after primary percutaneous coronary intervention for acute myocardial infarction: the CADILLAC risk score.","volume":"45","author":"Halkin","year":"2005","journal-title":"J Am Coll Cardiol"},{"key":"R5-20250502","doi-asserted-by":"crossref","first-page":"387","DOI":"10.1016\/S0735-1097(02)01980-0","article-title":"Correlates of procedural complications and a simple integer risk score for percutaneous coronary intervention.","volume":"40","author":"Singh","year":"2002","journal-title":"J Am Coll Cardiol"},{"key":"R6-20250502","doi-asserted-by":"crossref","first-page":"1514","DOI":"10.1213\/ANE.0000000000001541","article-title":"Development of a pediatric risk assessment score to predict perioperative mortality in children undergoing noncardiac surgery.","volume":"124","author":"Nasr","year":"2017","journal-title":"Anesth Analg"},{"key":"R7-20250502","doi-asserted-by":"crossref","first-page":"354","DOI":"10.1016\/j.jvs.2012.08.120","article-title":"Derivation and validation of a practical risk score for prediction of mortality after open repair of ruptured abdominal aortic aneurysms in a US regional cohort and comparison to existing scoring systems.","volume":"57","author":"Robinson","year":"2013","journal-title":"J Vasc Surg"},{"key":"R8-20250502","doi-asserted-by":"crossref","first-page":"2752","DOI":"10.3389\/fmicb.2019.02752","article-title":"Clinical features predicting mortality risk in patients with viral pneumonia: the MuLBSTA score.","volume":"10","author":"Guo","year":"2019","journal-title":"Front Microbiol"},{"key":"R9-20250502","doi-asserted-by":"crossref","first-page":"e157","DOI":"10.1016\/S2352-3026(17)30026-1","article-title":"Derivation and external validation of the PLASMIC score for rapid assessment of adults with thrombotic microangiopathies: a cohort study.","volume":"4","author":"Bendapudi","year":"2017","journal-title":"Lancet Haematol"},{"key":"R10-20250502","doi-asserted-by":"crossref","first-page":"127","DOI":"10.1002\/sim.2331","article-title":"Dichotomizing continuous predictors in multiple regression: a bad idea.","volume":"25","author":"Royston","year":"2006","journal-title":"Stat Med"},{"key":"R11-20250502","doi-asserted-by":"crossref","first-page":"829","DOI":"10.1093\/jnci\/86.11.829","article-title":"Dangers of using \u201coptimal\u201d cutpoints in the evaluation of prognostic factors.","volume":"86","author":"Altman","year":"1994","journal-title":"J Natl Cancer Inst"},{"key":"R12-20250502","doi-asserted-by":"crossref","first-page":"261","DOI":"10.2307\/2347432","article-title":"Scaling and rounding regression coefficients to integers.","volume":"42","author":"Cole","year":"1993","journal-title":"Appl Stat"},{"key":"R13-20250502","doi-asserted-by":"crossref","first-page":"111","DOI":"10.1097\/00042307-200303000-00005","article-title":"Nomograms are superior to staging and risk grouping systems for identifying high-risk patients: preoperative application in prostate cancer.","volume":"13","author":"Kattan","year":"2003","journal-title":"Curr Opin Urol"},{"key":"R14-20250502","doi-asserted-by":"crossref","first-page":"361","DOI":"10.1002\/(SICI)1097-0258(19960229)15:4<361::AID-SIM168>3.0.CO;2-4","article-title":"Multivariable prognostic models: issues in developing models, evaluating assumptions and adequacy, and measuring and reducing errors.","volume":"15","author":"Harrell","year":"1996","journal-title":"Stat Med"},{"key":"R15-20250502","doi-asserted-by":"crossref","first-page":"e0007","DOI":"10.1097\/CCE.0000000000000007","article-title":"Early prediction of prognosis in elderly acute stroke patients.","volume":"1","author":"Bautista","year":"2019","journal-title":"Crit Care Explor"},{"key":"R16-20250502","doi-asserted-by":"crossref","first-page":"219","DOI":"10.1097\/ALN.0000000000000334","article-title":"Prospective external validation of a predictive score for postoperative pulmonary complications.","volume":"121","author":"Mazo","year":"2014","journal-title":"Anesthesiology"},{"key":"R17-20250502","doi-asserted-by":"crossref","first-page":"32","DOI":"10.1002\/1097-0142(1950)3:1<32::AID-CNCR2820030106>3.0.CO;2-3","article-title":"Index for rating diagnostic tests.","volume":"3","author":"Youden","year":"1950","journal-title":"Cancer"},{"key":"R18-20250502","doi-asserted-by":"crossref","first-page":"820","DOI":"10.1213\/ANE.0000000000003680","article-title":"Identifying the best cut-point for a biomarker, or not.","volume":"127","author":"Mascha","year":"2018","journal-title":"Anesth Analg"},{"key":"R19-20250502","doi-asserted-by":"crossref","first-page":"840","DOI":"10.1213\/ANE.0000000000003341","article-title":"Harmful or physiologic: diagnosing fibrinolysis shutdown in a trauma cohort with rotational thromboelastometry.","volume":"127","author":"Gomez-Builes","year":"2018","journal-title":"Anesth Analg"},{"key":"R20-20250502","doi-asserted-by":"crossref","first-page":"29","DOI":"10.1148\/radiology.143.1.7063747","article-title":"The meaning and use of the area under a receiver operating characteristic (ROC) curve.","volume":"143","author":"Hanley","year":"1982","journal-title":"Radiology"},{"key":"R21-20250502","doi-asserted-by":"crossref","first-page":"7","DOI":"10.1186\/s41512-018-0029-2","article-title":"The index of prediction accuracy: an intuitive measure useful for evaluating risk prediction models.","volume":"2","author":"Kattan","year":"2018","journal-title":"Diagn Progn Res"},{"key":"R22-20250502","doi-asserted-by":"crossref","first-page":"1385","DOI":"10.1007\/s00180-012-0344-y","article-title":"Linear latent variable models: the lava-package.","volume":"28","author":"Holst","year":"2013","journal-title":"Comput Stat"},{"key":"R23-20250502","doi-asserted-by":"crossref","first-page":"440","DOI":"10.32614\/RJ-2017-062","article-title":"riskRegression: predicting the risk of an event using Cox regression models.","volume":"9","author":"Ozenne","year":"2017","journal-title":"R J"},{"key":"R29-20250502","doi-asserted-by":"crossref","first-page":"93","DOI":"10.1016\/j.jclinepi.2019.09.016","article-title":"Validation of clinical prediction models: what does the \u201ccalibration slope\u201d really measure?","volume":"118","author":"Stevens","year":"2020","journal-title":"J Clin Epidemiol"},{"key":"R30-20250502","doi-asserted-by":"crossref","first-page":"401","DOI":"10.1111\/bju.14640","article-title":"Guidelines for reporting of statistics for clinical research in urology.","volume":"123","author":"Assel","year":"2019","journal-title":"BJU Int"},{"key":"R31-20250502","doi-asserted-by":"crossref","first-page":"267","DOI":"10.1111\/j.2517-6161.1996.tb02080.x","article-title":"Regression shrinkage and selection via the LASSO.","volume":"58","author":"Tibshirani","year":"1996","journal-title":"J R Stat Soc Ser B Methodol"},{"key":"R32-20250502","doi-asserted-by":"crossref","first-page":"730","DOI":"10.1214\/009053604000001147","article-title":"Spike and slab variable selection: frequentist and Bayesian strategies.","volume":"33","author":"Ishwaran","year":"2005","journal-title":"Ann Stat"},{"key":"R33-20250502","doi-asserted-by":"crossref","first-page":"179","DOI":"10.1080\/01621459.1997.10473615","article-title":"Bayesian model averaging for linear regression models.","volume":"92","author":"Raftery","year":"1997","journal-title":"J Am Stat Assoc"},{"key":"R35-20250502","doi-asserted-by":"crossref","first-page":"3967","DOI":"10.1200\/JCO.2005.05.3884","article-title":"Postoperative nomogram predicting risk of recurrence after radical cystectomy for bladder cancer.","volume":"24","author":"Bochner","year":"2006","journal-title":"J Clin Oncol Off J Am Soc Clin Oncol"},{"key":"R36-20250502","doi-asserted-by":"crossref","first-page":"84","DOI":"10.1016\/j.urology.2005.07.013","article-title":"Development of nomogram to predict acute urinary retention or surgical intervention, with or without dutasteride therapy, in men with benign prostatic hyperplasia.","volume":"67","author":"Slawin","year":"2006","journal-title":"Urology"},{"key":"R37-20250502","doi-asserted-by":"crossref","first-page":"1140","DOI":"10.1245\/ASO.2003.03.015","article-title":"A nomogram for predicting the likelihood of additional nodal metastases in breast cancer patients with a positive sentinel node biopsy.","volume":"10","author":"Van Zee","year":"2003","journal-title":"Ann Surg Oncol"},{"key":"R41-20250502","doi-asserted-by":"crossref","first-page":"23","DOI":"10.1053\/j.seminoncol.2009.12.003","article-title":"What is a real nomogram?","volume":"37","author":"Kattan","year":"2010","journal-title":"Semin Oncol"},{"key":"R42-20250502","doi-asserted-by":"crossref","first-page":"774","DOI":"10.1016\/S0895-4356(01)00341-9","article-title":"Internal validation of predictive models: efficiency of some procedures for logistic regression analysis.","volume":"54","author":"Steyerberg","year":"2001","journal-title":"J Clin Epidemiol"},{"key":"R43-20250502","doi-asserted-by":"crossref","first-page":"441","DOI":"10.1016\/S0895-4356(03)00047-7","article-title":"Internal and external validation of predictive models: a simulation study of bias and precision in small samples.","volume":"56","author":"Steyerberg","year":"2003","journal-title":"J Clin Epidemiol"},{"key":"R44-20250502","doi-asserted-by":"crossref","first-page":"1142","DOI":"10.1016\/j.eururo.2014.11.025","article-title":"Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis (TRIPOD): the TRIPOD statement.","volume":"67","author":"Collins","year":"2015","journal-title":"Eur Urol"}],"container-title":["Anesthesia &amp; Analgesia"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.ovid.com\/10.1213\/ANE.0000000000005362","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,8,7]],"date-time":"2026-08-07T14:25:58Z","timestamp":1786112758000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.ovid.com\/10.1213\/ANE.0000000000005362"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,1,15]]},"references-count":35,"journal-issue":{"issue":"6","published-print":{"date-parts":[[2021]]}},"URL":"https:\/\/doi.org\/10.1213\/ane.0000000000005362","relation":{},"ISSN":["0003-2999"],"issn-type":[{"value":"0003-2999","type":"print"}],"subject":[],"published":{"date-parts":[[2021,1,15]]}}}