{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,7]],"date-time":"2026-09-07T19:14:42Z","timestamp":1788808482583,"version":"build-2803163510"},"reference-count":38,"publisher":"MDPI AG","issue":"2","license":[{"start":{"date-parts":[[2023,1,31]],"date-time":"2023-01-31T00:00:00Z","timestamp":1675123200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["JPM"],"abstract":"<jats:p>Although several models for the prediction of surgical complications after primary total hip or total knee replacement (THA and TKA, respectively) are available, only a few models have been externally validated. The aim of this study was to externally validate four previously developed models for the prediction of surgical complications in people considering primary THA or TKA. We included 2614 patients who underwent primary THA or TKA in secondary care between 2017 and 2020. Individual predicted probabilities of the risk for surgical complication per outcome (i.e., surgical site infection, postoperative bleeding, delirium, and nerve damage) were calculated for each model. The discriminative performance of patients with and without the outcome was assessed with the area under the receiver operating characteristic curve (AUC), and predictive performance was assessed with calibration plots. The predicted risk for all models varied between &lt;0.01 and 33.5%. Good discriminative performance was found for the model for delirium with an AUC of 84% (95% CI of 0.82\u20130.87). For all other outcomes, poor discriminative performance was found; 55% (95% CI of 0.52\u20130.58) for the model for surgical site infection, 61% (95% CI of 0.59\u20130.64) for the model for postoperative bleeding, and 57% (95% CI of 0.53\u20130.61) for the model for nerve damage. Calibration of the model for delirium was moderate, resulting in an underestimation of the actual probability between 2 and 6%, and exceeding 8%. Calibration of all other models was poor. Our external validation of four internally validated prediction models for surgical complications after THA and TKA demonstrated a lack of predictive accuracy when applied in another Dutch hospital population, with the exception of the model for delirium. This model included age, the presence of a heart disease, and the presence of a disease of the central nervous system as predictor variables. We recommend that clinicians use this simple and straightforward delirium model during preoperative counselling, shared decision-making, and early delirium precautionary interventions.<\/jats:p>","DOI":"10.3390\/jpm13020277","type":"journal-article","created":{"date-parts":[[2023,2,1]],"date-time":"2023-02-01T02:52:02Z","timestamp":1675219922000},"page":"277","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":5,"title":["External Validation of Prediction Models for Surgical Complications in People Considering Total Hip or Knee Arthroplasty Was Successful for Delirium but Not for Surgical Site Infection, Postoperative Bleeding, and Nerve Damage: A Retrospective Cohort Study"],"prefix":"10.3390","volume":"13","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-6940-3367","authenticated-orcid":false,"given":"Lieke","family":"Sweerts","sequence":"first","affiliation":[{"name":"Department of Orthopaedics, Radboud Institute for Health Sciences, Radboud University Medical Center, 6500 HB Nijmegen, The Netherlands"},{"name":"IQ Healthcare, Radboud Institute for Health Sciences, Radboud University Medical Center, 6500 HB Nijmegen, The Netherlands"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2589-8462","authenticated-orcid":false,"given":"Pepijn W.","family":"Dekkers","sequence":"additional","affiliation":[{"name":"Department of Orthopaedics, Radboud Institute for Health Sciences, Radboud University Medical Center, 6500 HB Nijmegen, The Netherlands"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Philip J.","family":"van der Wees","sequence":"additional","affiliation":[{"name":"IQ Healthcare, Radboud Institute for Health Sciences, Radboud University Medical Center, 6500 HB Nijmegen, The Netherlands"},{"name":"Department of Rehabilitation, Radboud Institute for Health Sciences, Radboud University Medical Center, 6500 HB Nijmegen, The Netherlands"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Job L. C.","family":"van Susante","sequence":"additional","affiliation":[{"name":"Department of Orthopedics, Rijnstate Hospital, 6800 TA Arnhem, The Netherlands"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1478-2619","authenticated-orcid":false,"given":"Lex D.","family":"de Jong","sequence":"additional","affiliation":[{"name":"Department of Orthopedics, Rijnstate Hospital, 6800 TA Arnhem, The Netherlands"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2103-419X","authenticated-orcid":false,"given":"Thomas J.","family":"Hoogeboom","sequence":"additional","affiliation":[{"name":"IQ Healthcare, Radboud Institute for Health Sciences, Radboud University Medical Center, 6500 HB Nijmegen, The Netherlands"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sebastiaan A. W.","family":"van de Groes","sequence":"additional","affiliation":[{"name":"Department of Orthopaedics, Radboud Institute for Health Sciences, Radboud University Medical Center, 6500 HB Nijmegen, The Netherlands"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2023,1,31]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"e021784","DOI":"10.1136\/bmjopen-2018-021784","article-title":"Impact of comorbid conditions on outcomes of hip and knee replacement surgery: A systematic review and meta-analysis","volume":"8","author":"Podmore","year":"2018","journal-title":"BMJ Open"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"e062065","DOI":"10.1136\/bmjopen-2022-062065","article-title":"Development of prediction models for complications after primary total hip and knee arthroplasty: A single-centre retrospective cohort study in the Netherlands","volume":"12","author":"Sweerts","year":"2022","journal-title":"BMJ Open"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"241","DOI":"10.1097\/CORR.0000000000001078","article-title":"How Accurate Are the Surgical Risk Preoperative Assessment System (SURPAS) Universal Calculators in Total Joint Arthroplasty?","volume":"478","author":"Trickey","year":"2020","journal-title":"Clin. Orthop. Relat. Res."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"452","DOI":"10.1097\/CORR.0000000000000601","article-title":"Can Machine Learning Methods Produce Accurate and Easy-to-use Prediction Models of 30-day Complications and Mortality After Knee or Hip Arthroplasty?","volume":"477","author":"Harris","year":"2019","journal-title":"Clin. Orthop. Relat. Res."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"1539","DOI":"10.1016\/j.arth.2017.12.003","article-title":"Prediction Models for 30-Day Mortality and Complications After Total Knee and Hip Arthroplasties for Veteran Health Administration Patients with Osteoarthritis","volume":"33","author":"Harris","year":"2018","journal-title":"J. Arthroplast."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"833","DOI":"10.1016\/j.jamcollsurg.2013.07.385","article-title":"Development and evaluation of the universal ACS NSQIP surgical risk calculator: A decision aid and informed consent tool for patients and surgeons","volume":"217","author":"Bilimoria","year":"2013","journal-title":"J. Am. Coll. Surg."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"1774","DOI":"10.1002\/bjs.9638","article-title":"Development and validation of the Surgical Outcome Risk Tool (SORT)","volume":"101","author":"Protopapa","year":"2014","journal-title":"Br. J. Surg."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"23","DOI":"10.1097\/SLA.0000000000001678","article-title":"Surgical Risk Preoperative Assessment System (SURPAS): III. Accurate Preoperative Prediction of 8 Adverse Outcomes Using 8 Predictor Variables","volume":"264","author":"Meguid","year":"2016","journal-title":"Ann. Surg."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"574","DOI":"10.1007\/s11999-012-2605-z","article-title":"Estimating risk in Medicare patients with THA: An electronic risk calculator for periprosthetic joint infection and mortality","volume":"471","author":"Bozic","year":"2013","journal-title":"Clin. Orthop. Relat. Res."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"777","DOI":"10.2106\/JBJS.16.01435","article-title":"Development and Evaluation of a Preoperative Risk Calculator for Periprosthetic Joint Infection Following Total Joint Arthroplasty","volume":"100","author":"Tan","year":"2018","journal-title":"JBJS"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"104217","DOI":"10.1016\/j.ijmedinf.2020.104217","article-title":"JointCalc: A web-based personalised patient decision support tool for joint replacement","volume":"142","author":"Zotov","year":"2020","journal-title":"Int. J. Med. Inform."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"3965","DOI":"10.1007\/s11999-015-4506-4","article-title":"Risk Calculators Predict Failures of Knee and Hip Arthroplasties: Findings from a Large Health Maintenance Organization","volume":"473","author":"Paxton","year":"2015","journal-title":"Clin. Orthop. Relat. Res."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"693","DOI":"10.1016\/j.arth.2020.08.004","article-title":"Development of a Preoperative Risk Calculator for Reinfection Following Revision Surgery for Periprosthetic Joint Infection","volume":"36","author":"Klemt","year":"2021","journal-title":"J. Arthroplast."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"1522","DOI":"10.2106\/JBJS.15.00988","article-title":"Development and Validation of a Preoperative Surgical Site Infection Risk Score for Primary or Revision Knee and Hip Arthroplasty","volume":"98","author":"Everhart","year":"2016","journal-title":"JBJS"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"1692","DOI":"10.1016\/j.arth.2015.05.004","article-title":"Predicting Infections After Total Joint Arthroplasty Using a Prescription Based Comorbidity Measure","volume":"30","author":"Inacio","year":"2015","journal-title":"J. Arthroplast."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"5","DOI":"10.1016\/j.arth.2015.01.057","article-title":"Can the American College of Surgeons Risk Calculator Predict 30-Day Complications After Knee and Hip Arthroplasty?","volume":"30","author":"Edelstein","year":"2015","journal-title":"J. Arthroplast."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Steyerberg, E.W., Moons, K.G., van der Windt, D.A., Hayden, J.A., Perel, P., Schroter, S., Riley, R.D., Hemingway, H., and Altman, D.G. (2013). Prognosis Research Strategy (PROGRESS) 3: Prognostic model research. PLoS Med., 10.","DOI":"10.1371\/journal.pmed.1001381"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"1738","DOI":"10.1017\/S0950268817000486","article-title":"Systematic review of risk prediction scores for surgical site infection or periprosthetic joint infection following joint arthroplasty","volume":"145","author":"Kunutsor","year":"2017","journal-title":"Epidemiol. Infect."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"56","DOI":"10.1016\/j.ijmedinf.2016.12.001","article-title":"Factors influencing implementation success of guideline-based clinical decision support systems: A systematic review and gaps analysis","volume":"98","author":"Kilsdonk","year":"2017","journal-title":"Int. J. Med. Inform."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"3070","DOI":"10.1007\/s11999-010-1399-0","article-title":"Risk of Subsequent Revision after Primary and Revision Total Joint Arthroplasty","volume":"468","author":"Ong","year":"2010","journal-title":"Clin. Orthop. Relat. Res."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"498","DOI":"10.3109\/17453674.2015.1028307","article-title":"More than 95% completeness of reported procedures in the population-based Dutch Arthroplasty Register","volume":"86","author":"Denissen","year":"2015","journal-title":"Acta Orthop."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"W1","DOI":"10.7326\/M14-0698","article-title":"Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis (TRIPOD): Explanation and elaboration","volume":"162","author":"Moons","year":"2015","journal-title":"Ann. Intern. Med."},{"key":"ref_23","unstructured":"CTcue (2022, July 12). Empowering Healthcare with Real-World Evidence. Available online: https:\/\/ctcue.com\/."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"159","DOI":"10.2307\/2529310","article-title":"The measurement of observer agreement for categorical data","volume":"33","author":"Landis","year":"1977","journal-title":"Biometrics"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"710","DOI":"10.1093\/aje\/kwk052","article-title":"Relaxing the rule of ten events per variable in logistic and Cox regression","volume":"165","author":"Vittinghoff","year":"2007","journal-title":"Am. J. Epidemiol."},{"key":"ref_26","unstructured":"Van Arkel, E., van der Kraan, J., Hageman, M., and Venhorst, K. (2022, July 12). Consultkaart Artrose in de heup. Available online: https:\/\/cdn.nimbu.io\/s\/yba55wt\/assets\/20161219_CK_Artrose-in-de-heup.pdf."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Steyerberg, E.W. (2009). Clinical Prediction Models, Springer.","DOI":"10.1007\/978-0-387-77244-8"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"128","DOI":"10.1097\/EDE.0b013e3181c30fb2","article-title":"Assessing the performance of prediction models: A framework for traditional and novel measures","volume":"21","author":"Steyerberg","year":"2010","journal-title":"Epidemiology"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Hosmer, D.W., Lemeshow, S., and Sturdivant, R. (2013). Applied Logistic Regression, Wiley. [3rd ed.].","DOI":"10.1002\/9781118548387"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"1373","DOI":"10.1016\/S0895-4356(96)00236-3","article-title":"A simulation study of the number of events per variable in logistic regression analysis","volume":"49","author":"Peduzzi","year":"1996","journal-title":"J. Clin. Epidemiol."},{"key":"ref_31","unstructured":"R Development Core Team (2022, July 27). R: A language and environment for statistical computing, R Foundation for Statistical Computing. Available online: http:\/\/www.R-project.org."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"25","DOI":"10.1016\/j.jclinepi.2014.09.007","article-title":"External validation of new risk prediction models is infrequent and reveals worse prognostic discrimination","volume":"68","author":"Siontis","year":"2015","journal-title":"J. Clin. Epidemiol."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"423","DOI":"10.1111\/1471-0528.14170","article-title":"Key steps and common pitfalls in developing and validating risk models","volume":"124","author":"Wynants","year":"2017","journal-title":"BJOG"},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"S574","DOI":"10.21037\/jtd.2019.01.25","article-title":"Developing prediction models for clinical use using logistic regression: An overview","volume":"11","author":"Shipe","year":"2019","journal-title":"J. Thorac. Dis."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"691","DOI":"10.1136\/heartjnl-2011-301247","article-title":"Risk prediction models: II. External validation, model updating, and impact assessment","volume":"98","author":"Moons","year":"2012","journal-title":"Heart"},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"644","DOI":"10.1002\/cpt.1966","article-title":"An Electronic Health Record Text Mining Tool to Collect Real-World Drug Treatment Outcomes: A Validation Study in Patients with Metastatic Renal Cell Carcinoma","volume":"108","author":"Guchelaar","year":"2020","journal-title":"Clin. Pharmacol. Ther."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"268","DOI":"10.1016\/j.arth.2013.06.004","article-title":"Pharmacologic Risk Factors for Post-Operative Delirium in Total Joint Arthroplasty Patients: A Case\u2013Control Study","volume":"29","author":"Nandi","year":"2014","journal-title":"J. Arthroplast."},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Kitsis, P., Zisimou, T., Gkiatas, I., Kostas-Agnantis, I., Gelalis, I., Korompilias, A., and Pakos, E. (2022). Postoperative Delirium and Postoperative Cognitive Dysfunction in Patients with Elective Hip or Knee Arthroplasty: A Narrative Review of the Literature. Life, 12.","DOI":"10.3390\/life12020314"}],"container-title":["Journal of Personalized Medicine"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2075-4426\/13\/2\/277\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T18:20:00Z","timestamp":1760120400000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2075-4426\/13\/2\/277"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,1,31]]},"references-count":38,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2023,2]]}},"alternative-id":["jpm13020277"],"URL":"https:\/\/doi.org\/10.3390\/jpm13020277","relation":{},"ISSN":["2075-4426"],"issn-type":[{"value":"2075-4426","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,1,31]]}}}