{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,24]],"date-time":"2026-10-24T02:40:34Z","timestamp":1792809634978,"version":"4.5.0"},"reference-count":64,"publisher":"Oxford University Press (OUP)","issue":"6","funder":[{"DOI":"10.13039\/100000092","name":"National Library of Medicine","doi-asserted-by":"publisher","award":["5T15LM007450-15"],"award-info":[{"award-number":["5T15LM007450-15"]}],"id":[{"id":"10.13039\/100000092","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000092","name":"National Library of Medicine","doi-asserted-by":"publisher","award":["1R21LM011664-01"],"award-info":[{"award-number":["1R21LM011664-01"]}],"id":[{"id":"10.13039\/100000092","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Veterans Health Administration","award":["VA HSR&D CDA-08-020"],"award-info":[{"award-number":["VA HSR&D CDA-08-020"]}]},{"name":"Veterans Health Administration","award":["VA HSR&D IIR 11-292"],"award-info":[{"award-number":["VA HSR&D IIR 11-292"]}]},{"name":"Veterans Health Administration","award":["VA HSR&D IIR 13-052"],"award-info":[{"award-number":["VA HSR&D IIR 13-052"]}]},{"name":"Veterans Health Administration","award":["VA HSR&D IIR 13-073"],"award-info":[{"award-number":["VA HSR&D IIR 13-073"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2017,11,1]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:sec>\n                    <jats:title>Objective<\/jats:title>\n                    <jats:p>Predictive analytics create opportunities to incorporate personalized risk estimates into clinical decision support. Models must be well calibrated to support decision-making, yet calibration deteriorates over time. This study explored the influence of modeling methods on performance drift and connected observed drift with data shifts in the patient population.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Materials and Methods<\/jats:title>\n                    <jats:p>Using 2003 admissions to Department of Veterans Affairs hospitals nationwide, we developed 7 parallel models for hospital-acquired acute kidney injury using common regression and machine learning methods, validating each over 9 subsequent years.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>Discrimination was maintained for all models. Calibration declined as all models increasingly overpredicted risk. However, the random forest and neural network models maintained calibration across ranges of probability, capturing more admissions than did the regression models. The magnitude of overprediction increased over time for the regression models while remaining stable and small for the machine learning models. Changes in the rate of acute kidney injury were strongly linked to increasing overprediction, while changes in predictor-outcome associations corresponded with diverging patterns of calibration drift across methods.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusions<\/jats:title>\n                    <jats:p>Efficient and effective updating protocols will be essential for maintaining accuracy of, user confidence in, and safety of personalized risk predictions to support decision-making. Model updating protocols should be tailored to account for variations in calibration drift across methods and respond to periods of rapid performance drift rather than be limited to regularly scheduled annual or biannual intervals.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.1093\/jamia\/ocx030","type":"journal-article","created":{"date-parts":[[2017,3,13]],"date-time":"2017-03-13T16:36:05Z","timestamp":1489422965000},"page":"1052-1061","source":"Crossref","is-referenced-by-count":312,"title":["Calibration drift in regression and machine learning models for acute kidney injury"],"prefix":"10.1093","volume":"24","author":[{"given":"Sharon E","family":"Davis","sequence":"first","affiliation":[{"name":"Department of Biomedical Informatics, Vanderbilt University School of Medicine, Nashville, TN, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Thomas A","family":"Lasko","sequence":"additional","affiliation":[{"name":"Department of Biomedical Informatics, Vanderbilt University School of Medicine, Nashville, TN, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Guanhua","family":"Chen","sequence":"additional","affiliation":[{"name":"Department of Biostatistics, Vanderbilt University School of Medicine"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Edward D","family":"Siew","sequence":"additional","affiliation":[{"name":"Geriatric Research Education and Clinical Care Service, VA Tennessee Valley Healthcare System, Nashville, TN, USA"},{"name":"Division of Nephrology, Vanderbilt University School of Medicine, Vanderbilt Center for Kidney Disease and Integrated Program for AKI, Nashville, TN, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Michael E","family":"Matheny","sequence":"additional","affiliation":[{"name":"Department of Biomedical Informatics, Vanderbilt University School of Medicine, Nashville, TN, USA"},{"name":"Department of Biostatistics, Vanderbilt University School of Medicine"},{"name":"Geriatric Research Education and Clinical Care Service, VA Tennessee Valley Healthcare System, Nashville, TN, USA"},{"name":"Division of General Internal Medicine, Vanderbilt University School of Medicine"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"286","published-online":{"date-parts":[[2017,3,31]]},"reference":[{"issue":"7","key":"2020110612445257300_ocx030-B1","doi-asserted-by":"crossref","first-page":"1148","DOI":"10.1377\/hlthaff.2014.0352","article-title":"Implementing electronic health care predictive analytics: considerations and challenges","volume":"33","author":"Amarasingham","year":"2014","journal-title":"Health Affairs."},{"key":"2020110612445257300_ocx030-B2","doi-asserted-by":"crossref","first-page":"567","DOI":"10.1146\/annurev.bioeng.8.061505.095842","article-title":"Prognosis in critical care","volume":"8","author":"Ohno-Machado","year":"2006","journal-title":"Ann Rev Biomed Engineering."},{"issue":"9","key":"2020110612445257300_ocx030-B3","doi-asserted-by":"crossref","first-page":"691","DOI":"10.1136\/heartjnl-2011-301247","article-title":"Risk prediction models: II. External validation, model updating, and impact assessment","volume":"98","author":"Moons","year":"2012","journal-title":"Heart."},{"issue":"6","key":"2020110612445257300_ocx030-B4","doi-asserted-by":"crossref","first-page":"639","DOI":"10.1177\/0272989X10364246","article-title":"Development of inpatient risk stratification models of acute kidney injury for use in electronic health records","volume":"30","author":"Matheny","year":"2010","journal-title":"Med Decision Making."},{"issue":"15","key":"2020110612445257300_ocx030-B5","doi-asserted-by":"crossref","first-page":"1688","DOI":"10.1001\/jama.2011.1515","article-title":"Risk prediction models for hospital readmission: a systematic review","volume":"306","author":"Kansagara","year":"2011","journal-title":"JAMA."},{"issue":"2","key":"2020110612445257300_ocx030-B6","doi-asserted-by":"crossref","first-page":"e1001381","DOI":"10.1371\/journal.pmed.1001381","article-title":"Prognosis Research Strategy (PROGRESS) 3: prognostic model research","volume":"10","author":"Steyerberg","year":"2013","journal-title":"PLoS Med."},{"issue":"11","key":"2020110612445257300_ocx030-B7","doi-asserted-by":"crossref","first-page":"1085","DOI":"10.1016\/j.jclinepi.2008.04.008","article-title":"Validation, updating and impact of clinical prediction rules: a review","volume":"61","author":"Toll","year":"2008","journal-title":"J Clin Epidemiol."},{"issue":"6","key":"2020110612445257300_ocx030-B8","doi-asserted-by":"crossref","first-page":"1146","DOI":"10.1093\/ejcts\/ezs584","article-title":"Dynamic trends in cardiac surgery: why the logistic euroscore is no longer suitable for contemporary cardiac surgery and implications for future risk models","volume":"43","author":"Hickey","year":"2013","journal-title":"Eur J Cardio-thoracic Surg."},{"issue":"1","key":"2020110612445257300_ocx030-B9","doi-asserted-by":"crossref","first-page":"40","DOI":"10.1007\/s00134-011-2390-2","article-title":"Effect of changes over time in the performance of a customized SAPS-II model on the quality of care assessment","volume":"38","author":"Minne","year":"2012","journal-title":"Intensive Care Med."},{"issue":"4","key":"2020110612445257300_ocx030-B10","doi-asserted-by":"crossref","first-page":"353","DOI":"10.3414\/ME11-02-0044","article-title":"Statistical process control for monitoring standardized mortality ratios of a classification tree model","volume":"51","author":"Minne","year":"2012","journal-title":"Methods Inf Med."},{"key":"2020110612445257300_ocx030-B11","doi-asserted-by":"crossref","first-page":"b606","DOI":"10.1136\/bmj.b606","article-title":"Prognosis and prognostic research: application and impact of prognostic models in clinical practice","volume":"338","author":"Moons","year":"2009","journal-title":"BMJ."},{"key":"2020110612445257300_ocx030-B12","doi-asserted-by":"crossref","first-page":"116","DOI":"10.1186\/1471-2253-14-116","article-title":"External validation of the Intensive Care National Audit & Research Centre (ICNARC) risk prediction model in critical care units in Scotland","volume":"14","author":"Harrison","year":"2014","journal-title":"BMC Anesthesiol."},{"issue":"6","key":"2020110612445257300_ocx030-B13","doi-asserted-by":"crossref","first-page":"980","DOI":"10.1177\/0310057X1204000609","article-title":"Performance of APACHE III over time in Australia and New Zealand: a retrospective cohort study","volume":"40","author":"Paul","year":"2012","journal-title":"Anaesthesia Intensive Care."},{"issue":"2","key":"2020110612445257300_ocx030-B14","doi-asserted-by":"crossref","first-page":"227","DOI":"10.1016\/j.ijcard.2010.02.005","article-title":"Risk-prediction models for mortality after coronary artery bypass surgery: application to individual patients","volume":"149","author":"Madan","year":"2011","journal-title":"Int J Cardiol."},{"issue":"1","key":"2020110612445257300_ocx030-B15","doi-asserted-by":"crossref","first-page":"1","DOI":"10.13063\/2327-9214.1163","article-title":"Consensus statement on electronic health predictive analytics: a guiding framework to address challenges","volume":"4","author":"Amarasingham","year":"2016","journal-title":"eGEMs."},{"issue":"7","key":"2020110612445257300_ocx030-B16","doi-asserted-by":"crossref","first-page":"651","DOI":"10.1001\/jama.2015.19417","article-title":"Integrating predictive analytics into high-value care: the dawn of precision delivery","volume":"315","author":"Parikh","year":"2016","journal-title":"JAMA."},{"issue":"16","key":"2020110612445257300_ocx030-B17","doi-asserted-by":"crossref","first-page":"1713","DOI":"10.1001\/jama.2016.4839","article-title":"Moving from clinical trials to precision medicine: the role for predictive modeling","volume":"315","author":"Pencina","year":"2016","journal-title":"JAMA."},{"key":"2020110612445257300_ocx030-B18","doi-asserted-by":"crossref","first-page":"537","DOI":"10.1146\/annurev.bioeng.8.061505.095802","article-title":"Machine learning for detection and diagnosis of disease","volume":"8","author":"Sajda","year":"2006","journal-title":"Ann Rev Biomed Engineering."},{"issue":"4","key":"2020110612445257300_ocx030-B19","doi-asserted-by":"crossref","first-page":"601","DOI":"10.1002\/bimj.201300297","article-title":"Risk prediction with machine learning and regression methods","volume":"56","author":"Steyerberg","year":"2014","journal-title":"Biometrical J. Biometrische Zeitschrift."},{"issue":"7","key":"2020110612445257300_ocx030-B20","doi-asserted-by":"crossref","first-page":"813","DOI":"10.1001\/jama.294.7.813","article-title":"Acute renal failure in critically ill patients: a multinational, multicenter study","volume":"294","author":"Uchino","year":"2005","journal-title":"JAMA."},{"issue":"2","key":"2020110612445257300_ocx030-B21","doi-asserted-by":"crossref","first-page":"192","DOI":"10.1097\/00003246-199602000-00003","article-title":"Acute renal failure in intensive care units \u2013 causes, outcome, and prognostic factors of hospital mortality: a prospective, multicenter study. French Study Group on Acute Renal Failure","volume":"24","author":"Brivet","year":"1996","journal-title":"Crit Care Med."},{"issue":"6","key":"2020110612445257300_ocx030-B22","doi-asserted-by":"crossref","first-page":"961","DOI":"10.1053\/j.ajkd.2008.11.034","article-title":"Long-term risk of mortality and other adverse outcomes after acute kidney injury: a systematic review and meta-analysis","volume":"53","author":"Coca","year":"2009","journal-title":"Am J Kidney Dis."},{"key":"2020110612445257300_ocx030-B23","first-page":"S16","article-title":"The spectrum of acute renal failure in the intensive care unit compared with that seen in other settings","volume":"66","author":"Liano","year":"1998","journal-title":"The Madrid Acute Renal Failure Study Group. Kidney Int Suppl."},{"issue":"5","key":"2020110612445257300_ocx030-B24","doi-asserted-by":"crossref","first-page":"1054","DOI":"10.1093\/jamia\/ocv051","article-title":"National Veterans Health Administration Inpatient Risk Stratification Models for Hospital-Acquired Acute Kidney Injury","volume":"22","author":"Cronin","year":"2015","journal-title":"J Am Med Inform Assoc."},{"issue":"2","key":"2020110612445257300_ocx030-B25","doi-asserted-by":"crossref","first-page":"168","DOI":"10.1016\/j.amjmed.2011.07.010","article-title":"A combined cardiorenal assessment for the prediction of acute kidney injury in lower respiratory tract infections","volume":"125","author":"Breidthardt","year":"2012","journal-title":"Am J Med."},{"issue":"6","key":"2020110612445257300_ocx030-B26","doi-asserted-by":"crossref","first-page":"1158","DOI":"10.1053\/j.jvca.2013.04.007","article-title":"Simplified clinical risk score to predict acute kidney injury after aortic surgery","volume":"27","author":"Kim","year":"2013","journal-title":"J Cardiothorac Vasc Anesth."},{"issue":"3","key":"2020110612445257300_ocx030-B27","doi-asserted-by":"crossref","first-page":"366","DOI":"10.1093\/icvts\/ivv162","article-title":"Cardiac surgery\u2013associated acute kidney injury: risk factors analysis and comparison of prediction models","volume":"21","author":"Kristovic","year":"2015","journal-title":"Interact Cardiovasc Thorac Surg."},{"issue":"19","key":"2020110612445257300_ocx030-B28","doi-asserted-by":"crossref","first-page":"1821","DOI":"10.1001\/jamainternmed.2013.9774","article-title":"A risk prediction score for kidney failure or mortality in rhabdomyolysis","volume":"173","author":"McMahon","year":"2013","journal-title":"JAMA Int Med."},{"issue":"6","key":"2020110612445257300_ocx030-B29","doi-asserted-by":"crossref","first-page":"1875","DOI":"10.1016\/j.jtcvs.2013.06.049","article-title":"Prediction of acute kidney injury within 30 days of cardiac surgery","volume":"147","author":"Ng","year":"2014","journal-title":"J Thoracic Cardiovasc Surgery."},{"issue":"8","key":"2020110612445257300_ocx030-B30","doi-asserted-by":"crossref","first-page":"e0136230","DOI":"10.1371\/journal.pone.0136230","article-title":"Clinical risk scoring models for prediction of acute kidney injury after living donor liver transplantation: a retrospective observational study","volume":"10","author":"Park","year":"2015","journal-title":"PloS One."},{"issue":"11","key":"2020110612445257300_ocx030-B31","doi-asserted-by":"crossref","first-page":"2618","DOI":"10.1007\/s00268-013-2159-6","article-title":"Novel prediction score including pre- and intraoperative parameters best predicts acute kidney injury after liver surgery","volume":"37","author":"Slankamenac","year":"2013","journal-title":"World J Surgery."},{"issue":"7","key":"2020110612445257300_ocx030-B32","doi-asserted-by":"crossref","first-page":"489","DOI":"10.1111\/nep.12092","article-title":"Derivation and validation of a prediction score for acute kidney injury in patients hospitalized with acute heart failure in a Chinese cohort","volume":"18","author":"Wang","year":"2013","journal-title":"Nephrology."},{"issue":"12","key":"2020110612445257300_ocx030-B33","doi-asserted-by":"crossref","first-page":"e82992","DOI":"10.1371\/journal.pone.0082992","article-title":"Risk factors for acute kidney injury in severe rhabdomyolysis","volume":"8","author":"Rodriguez","year":"2013","journal-title":"PloS One."},{"issue":"2","key":"2020110612445257300_ocx030-B34","doi-asserted-by":"crossref","first-page":"242","DOI":"10.1097\/BCR.0b013e318239cc24","article-title":"Predicting acute kidney injury among burn patients in the 21st century: a classification and regression tree analysis","volume":"33","author":"Schneider","year":"2012","journal-title":"J Burn Care Res."},{"issue":"5","key":"2020110612445257300_ocx030-B35","doi-asserted-by":"crossref","first-page":"R220","DOI":"10.1186\/cc13041","article-title":"Incidence, risk factors and prediction of post-operative acute kidney injury following cardiac surgery for active infective endocarditis: an observational study","volume":"17","author":"Legrand","year":"2013","journal-title":"Crit Care."},{"issue":"12","key":"2020110612445257300_ocx030-B36","doi-asserted-by":"crossref","first-page":"e002136","DOI":"10.1161\/JAHA.115.002136","article-title":"Acute kidney injury risk prediction in patients undergoing coronary angiography in a national Veterans Health Administration cohort with external validation","volume":"4","author":"Brown","year":"2015","journal-title":"J Am Heart Assoc."},{"issue":"22","key":"2020110612445257300_ocx030-B37","doi-asserted-by":"crossref","first-page":"2242","DOI":"10.1016\/j.jacc.2013.03.026","article-title":"A novel tool for reliable and accurate prediction of renal complications in patients undergoing percutaneous coronary intervention","volume":"61","author":"Gurm","year":"2013","journal-title":"J Am Coll Cardiol."},{"issue":"5","key":"2020110612445257300_ocx030-B38","doi-asserted-by":"crossref","first-page":"367","DOI":"10.1016\/j.jbi.2005.02.007","article-title":"Discrimination and calibration of mortality risk prediction models in interventional cardiology","volume":"38","author":"Matheny","year":"2005","journal-title":"J Biomed Inform."},{"issue":"2","key":"2020110612445257300_ocx030-B39","doi-asserted-by":"crossref","first-page":"263","DOI":"10.1136\/amiajnl-2011-000291","article-title":"Calibrating predictive model estimates to support personalized medicine","volume":"19","author":"Jiang","year":"2012","journal-title":"J Am Med Inform Assoc,"},{"issue":"2","key":"2020110612445257300_ocx030-B40","doi-asserted-by":"crossref","first-page":"162","DOI":"10.1177\/0272989X14547233","article-title":"Calibration of risk prediction models: impact on decision-analytic performance","volume":"35","author":"Van Calster","year":"2015","journal-title":"Med Decis Mak."},{"key":"2020110612445257300_ocx030-B41","doi-asserted-by":"crossref","DOI":"10.1007\/978-0-387-77244-8","volume-title":"Clinical Prediction Models: A Practical Approach to Development, Validation, and Updating","author":"Steyerberg","year":"2009"},{"issue":"1","key":"2020110612445257300_ocx030-B42","doi-asserted-by":"crossref","first-page":"128","DOI":"10.1097\/EDE.0b013e3181c30fb2","article-title":"Assessing the performance of prediction models: a framework for traditional and novel measures","volume":"21","author":"Steyerberg","year":"2010","journal-title":"Epidemiology."},{"issue":"5","key":"2020110612445257300_ocx030-B43","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1371\/journal.pmed.1001221","article-title":"Reporting and methods in clinical prediction research: a systematic review","volume":"9","author":"Bouwmeester","year":"2012","journal-title":"PLoS Med."},{"key":"2020110612445257300_ocx030-B44","doi-asserted-by":"crossref","first-page":"40","DOI":"10.1186\/1471-2288-14-40","article-title":"External validation of multivariable prediction models: a systematic review of methodological conduct and reporting","volume":"14","author":"Collins","year":"2014","journal-title":"BMC Med Res Methodol."},{"key":"2020110612445257300_ocx030-B45","doi-asserted-by":"crossref","first-page":"21","DOI":"10.1186\/1741-7015-8-21","article-title":"Reporting performance of prognostic models in cancer: a review","volume":"8","author":"Mallett","year":"2010","journal-title":"BMC Med."},{"issue":"3","key":"2020110612445257300_ocx030-B46","doi-asserted-by":"crossref","first-page":"308","DOI":"10.1177\/0310057X0203000307","article-title":"Prospective independent validation of APACHE III models in an Australian tertiary adult intensive care unit","volume":"30","author":"Cook","year":"2002","journal-title":"Anaesth Intensive Care."},{"issue":"1","key":"2020110612445257300_ocx030-B47","doi-asserted-by":"crossref","first-page":"74","DOI":"10.3414\/ME00-01-0052","article-title":"Probability machines: consistent probability estimation using nonparametric learning machines","volume":"51","author":"Malley","year":"2012","journal-title":"Methods Inform Med."},{"issue":"3","key":"2020110612445257300_ocx030-B48","doi-asserted-by":"crossref","first-page":"199","DOI":"10.1214\/ss\/1009213726","article-title":"Statistical modeling: the two cultures","volume":"16","author":"Breiman","year":"2001","journal-title":"Statistical Science."},{"issue":"11 Pt 2","key":"2020110612445257300_ocx030-B49","first-page":"828","article-title":"The Veterans Health Administration: quality, value, accountability, and information as transforming strategies for patient-centered care","volume":"10","author":"Perlin","year":"2004","journal-title":"Am J Managed Care."},{"issue":"4","key":"2020110612445257300_ocx030-B50","doi-asserted-by":"crossref","first-page":"c179","DOI":"10.1159\/000339789","article-title":"KDIGO clinical practice guidelines for acute kidney injury","volume":"120","author":"Khwaja","year":"2012","journal-title":"Nephron Clin Pract."},{"issue":"1","key":"2020110612445257300_ocx030-B51","doi-asserted-by":"crossref","first-page":"267","DOI":"10.1111\/j.2517-6161.1996.tb02080.x","article-title":"Regression shrinkage and selection via the lasso","volume":"58","author":"Tibshirani","year":"1996","journal-title":"J Royal Stat Soc Series B."},{"issue":"1","key":"2020110612445257300_ocx030-B52","doi-asserted-by":"crossref","first-page":"55","DOI":"10.1080\/00401706.1970.10488634","article-title":"Ridge regression: biased estimation for nonorthogonal problems","volume":"12","author":"Hoerl","year":"1970","journal-title":"Technometrics."},{"issue":"2","key":"2020110612445257300_ocx030-B53","doi-asserted-by":"crossref","first-page":"301","DOI":"10.1111\/j.1467-9868.2005.00503.x","article-title":"Regularization and variable selection via the elastic net","volume":"67","author":"Zou","year":"2005","journal-title":"J Royal Stat Soc Series B."},{"issue":"1","key":"2020110612445257300_ocx030-B54","doi-asserted-by":"crossref","first-page":"5","DOI":"10.1023\/A:1010933404324","article-title":"Random forests","volume":"45","author":"Breiman","year":"2001","journal-title":"Machine Learning."},{"key":"2020110612445257300_ocx030-B55","doi-asserted-by":"crossref","DOI":"10.1093\/oso\/9780198538493.001.0001","volume-title":"Neural Networks for Pattern Recognition","author":"Bishop","year":"1995"},{"key":"2020110612445257300_ocx030-B56","doi-asserted-by":"crossref","first-page":"163","DOI":"10.1201\/9781420089653.ch9","article-title":"Naive Bayes","volume-title":"The Top Ten Algorithms in Data Mining","author":"Hand","year":"2009"},{"issue":"1","key":"2020110612445257300_ocx030-B57","doi-asserted-by":"crossref","first-page":"29","DOI":"10.1148\/radiology.143.1.7063747","article-title":"The meaning and use of the area under a receiver operating characteristic (ROC) curve","volume":"143","author":"Hanley","year":"1982","journal-title":"Radiology."},{"key":"2020110612445257300_ocx030-B58","doi-asserted-by":"crossref","first-page":"167","DOI":"10.1016\/j.jclinepi.2015.12.005","article-title":"A calibration hierarchy for risk models was defined: from utopia to empirical data","volume":"74","author":"Van Calster","year":"2016","journal-title":"J Clin Epidemiol."},{"key":"2020110612445257300_ocx030-B59","doi-asserted-by":"crossref","first-page":"283","DOI":"10.1016\/j.jbi.2014.12.016","article-title":"A spline-based tool to assess and visualize the calibration of multiclass risk predictions","volume":"54","author":"Van Hoorde","year":"2015","journal-title":"J Biomed Inform."},{"issue":"3","key":"2020110612445257300_ocx030-B60","doi-asserted-by":"crossref","first-page":"279","DOI":"10.1016\/j.jclinepi.2014.06.018","article-title":"A new framework to enhance the interpretation of external validation studies of clinical prediction models","volume":"68","author":"Debray","year":"2015","journal-title":"J Clin Epidemiol."},{"issue":"4","key":"2020110612445257300_ocx030-B61","doi-asserted-by":"crossref","first-page":"417","DOI":"10.1111\/j.1467-9868.2010.00740.x","article-title":"Stability selection","volume":"72","author":"Meinshausen","year":"2010","journal-title":"J Royal Stat Soc Series B (Statistical Methodology)."},{"issue":"16","key":"2020110612445257300_ocx030-B62","doi-asserted-by":"crossref","first-page":"2567","DOI":"10.1002\/sim.1844","article-title":"Validation and updating of predictive logistic regression models: a study on sample size and shrinkage","volume":"23","author":"Steyerberg","year":"2004","journal-title":"Stat Med."},{"issue":"3","key":"2020110612445257300_ocx030-B63","doi-asserted-by":"crossref","first-page":"E1","DOI":"10.1177\/0272989X12439755","article-title":"Adaptation of clinical prediction models for application in local settings","volume":"32","author":"Kappen","year":"2012","journal-title":"Med Decis Mak."},{"issue":"1","key":"2020110612445257300_ocx030-B64","doi-asserted-by":"crossref","first-page":"76","DOI":"10.1016\/j.jclinepi.2007.04.018","article-title":"Updating methods improved the performance of a clinical prediction model in new patients","volume":"61","author":"Janssen","year":"2008","journal-title":"J Clin Epidemiol."}],"container-title":["Journal of the American Medical Informatics Association"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/academic.oup.com\/jamia\/article-pdf\/24\/6\/1052\/34149285\/ocx030.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"http:\/\/academic.oup.com\/jamia\/article-pdf\/24\/6\/1052\/34149285\/ocx030.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,6,22]],"date-time":"2024-06-22T19:22:07Z","timestamp":1719084127000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/jamia\/article\/24\/6\/1052\/3096776"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2017,3,31]]},"references-count":64,"journal-issue":{"issue":"6","published-online":{"date-parts":[[2017,3,31]]},"published-print":{"date-parts":[[2017,11,1]]}},"URL":"https:\/\/doi.org\/10.1093\/jamia\/ocx030","relation":{},"ISSN":["1067-5027","1527-974X"],"issn-type":[{"value":"1067-5027","type":"print"},{"value":"1527-974X","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2017,11]]},"published":{"date-parts":[[2017,3,31]]}}}