{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,21]],"date-time":"2026-05-21T03:49:50Z","timestamp":1779335390123,"version":"3.51.4"},"reference-count":32,"publisher":"Oxford University Press (OUP)","issue":"8","license":[{"start":{"date-parts":[[2018,2,10]],"date-time":"2018-02-10T00:00:00Z","timestamp":1518220800000},"content-version":"vor","delay-in-days":1,"URL":"http:\/\/creativecommons.org\/licenses\/by-nc\/4.0\/"}],"funder":[{"DOI":"10.13039\/100000062","name":"National Institute of Diabetes and Digestive and Kidney Diseases","doi-asserted-by":"publisher","award":["K25DK097279"],"award-info":[{"award-number":["K25DK097279"]}],"id":[{"id":"10.13039\/100000062","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Veterans Affairs Health Services Research and Development Career Development","award":["CDA 14-158"],"award-info":[{"award-number":["CDA 14-158"]}]},{"name":"Veterans Affairs Health Services Research and Development Career Development","award":["UL 1TR001117"],"award-info":[{"award-number":["UL 1TR001117"]}]},{"DOI":"10.13039\/100006108","name":"National Center for Advancing Translational Sciences","doi-asserted-by":"publisher","id":[{"id":"10.13039\/100006108","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000002","name":"National Institutes of Health","doi-asserted-by":"publisher","id":[{"id":"10.13039\/100000002","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000002","name":"NIH","doi-asserted-by":"publisher","id":[{"id":"10.13039\/100000002","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2018,8,1]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:sec><jats:title>Objective<\/jats:title><jats:p>As available data increases, so does the opportunity to develop risk scores on more refined patient populations. In this paper we assessed the ability to derive a risk score for a patient no-showing to a clinic visit.<\/jats:p><\/jats:sec><jats:sec><jats:title>Methods<\/jats:title><jats:p>Using data from 2\u2009264\u2009235 outpatient appointments we assessed the performance of models built across 14 different specialties and 55 clinics. We used regularized logistic regression models to fit and assess models built on the health system, specialty, and clinic levels. We evaluated fits based on their discrimination and calibration.<\/jats:p><\/jats:sec><jats:sec><jats:title>Results<\/jats:title><jats:p>Overall, the results suggest that a relatively robust risk score for patient no-shows could be derived with an average C-statistic of 0.83 across clinic level models and strong calibration. Moreover, the clinic specific models, even with lower training set sizes, often performed better than the more general models. Examination of the individual models showed that risk factors had different degrees of predictability across the different specialties. Implementation of optimal modeling strategies would lead to capturing an additional 4819 no-shows per-year.<\/jats:p><\/jats:sec><jats:sec><jats:title>Conclusion<\/jats:title><jats:p>Overall, this work highlights both the opportunity for and the importance of leveraging the available electronic health record data to develop more refined risk models.<\/jats:p><\/jats:sec>","DOI":"10.1093\/jamia\/ocy002","type":"journal-article","created":{"date-parts":[[2018,1,7]],"date-time":"2018-01-07T20:07:35Z","timestamp":1515355655000},"page":"924-930","source":"Crossref","is-referenced-by-count":43,"title":["Designing risk prediction models for ambulatory no-shows across different specialties and clinics"],"prefix":"10.1093","volume":"25","author":[{"given":"Xiruo","family":"Ding","sequence":"first","affiliation":[{"name":"Department of Biostatistics and Bioinformatics, Duke University, Durham, North Carolina, 27710, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Ziad F","family":"Gellad","sequence":"additional","affiliation":[{"name":"Department of Medicine, Duke University, Durham, North Carolina, 27703, USA"},{"name":"Department of Medicine, Durham VA Medical Center, Durham, North Carolina, 27705, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"suffix":"III","given":"Chad","family":"Mather","sequence":"additional","affiliation":[{"name":"Department of Medicine, Duke University, Durham, North Carolina, 27703, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Pamela","family":"Barth","sequence":"additional","affiliation":[{"name":"Duke Health Technology Solutions, Duke University, Durham, North Carolina, 27713, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Eric G","family":"Poon","sequence":"additional","affiliation":[{"name":"Duke Health Technology Solutions, Duke University, Durham, North Carolina, 27713, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Mark","family":"Newman","sequence":"additional","affiliation":[{"name":"Department of Anesthesiology, University of Kentucky, Lexington, Kentucky, 40536, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Benjamin A","family":"Goldstein","sequence":"additional","affiliation":[{"name":"Department of Biostatistics and Bioinformatics, Duke University, Durham, North Carolina, 27710, USA"},{"name":"Center for Predictive Medicine, Duke Clinical Research Institute, Durham, North Carolina, 27705, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"286","published-online":{"date-parts":[[2018,2,9]]},"reference":[{"issue":"1","key":"2020110612223593600_ocy002-B1","doi-asserted-by":"crossref","first-page":"198","DOI":"10.1093\/jamia\/ocw042","article-title":"Opportunities and challenges in developing risk prediction models with electronic health records: a systematic review","volume":"24","author":"Goldstein","year":"2017","journal-title":"J Am Med Inform Assoc"},{"issue":"11","key":"2020110612223593600_ocy002-B2","doi-asserted-by":"crossref","first-page":"e112944","DOI":"10.1371\/journal.pone.0112944","article-title":"Risk prediction of emergency department revisit 30 days post discharge: a prospective study","volume":"9","author":"Hao","year":"2014","journal-title":"PLoS One"},{"issue":"10","key":"2020110612223593600_ocy002-B3","doi-asserted-by":"crossref","first-page":"e012705","DOI":"10.1136\/bmjopen-2016-012705","article-title":"Predicting frequent hospital admission risk in Singapore: a retrospective cohort study to investigate the impact of comorbidities, acute illness burden and social determinants of health","volume":"6","author":"Low","year":"2016","journal-title":"BMJ Open"},{"issue":"3","key":"2020110612223593600_ocy002-B4","doi-asserted-by":"crossref","first-page":"245","DOI":"10.1016\/j.jchf.2013.01.008","article-title":"Validated, electronic health record deployable prediction models for assessing patient risk of 30-day rehospitalization and mortality in older heart failure patients","volume":"1","author":"Eapen","year":"2013","journal-title":"JACC Heart Fail"},{"issue":"6","key":"2020110612223593600_ocy002-B5","doi-asserted-by":"crossref","first-page":"433","DOI":"10.5414\/CN107961","article-title":"Readmission after hospitalization for heart failure among patients with chronic kidney disease: a prediction model","volume":"80","author":"Perkins","year":"2013","journal-title":"Clin Nephrol"},{"key":"2020110612223593600_ocy002-B6","doi-asserted-by":"crossref","first-page":"65","DOI":"10.1186\/1472-6947-14-65","article-title":"Diagnosis-specific readmission risk prediction using electronic health data: a retrospective cohort study","volume":"14","author":"Hebert","year":"2014","journal-title":"BMC Med Inform Decis Mak"},{"issue":"8","key":"2020110612223593600_ocy002-B7","doi-asserted-by":"crossref","first-page":"976","DOI":"10.1177\/0272989X13478194","article-title":"Estimating the cost of no-shows and evaluating the effects of mitigation strategies","volume":"33","author":"Berg","year":"2013","journal-title":"Med Decis Mak"},{"issue":"1","key":"2020110612223593600_ocy002-B8","doi-asserted-by":"crossref","first-page":"13","DOI":"10.1186\/s12913-015-1243-z","article-title":"Prevalence, predictors and economic consequences of no-shows","volume":"16","author":"Kheirkhah","year":"2016","journal-title":"BMC Health Serv Res"},{"issue":"3","key":"2020110612223593600_ocy002-B9","doi-asserted-by":"crossref","first-page":"836","DOI":"10.4338\/ACI-2014-04-RA-0026","article-title":"Patient no-show predictive model development using multiple data sources for an effective overbooking approach","volume":"5","author":"Huang","year":"2014","journal-title":"Appl Clin Inf"},{"issue":"3","key":"2020110612223593600_ocy002-B10","doi-asserted-by":"crossref","first-page":"258","DOI":"10.1017\/S0022215106003653","article-title":"Nonattendance of adult otolaryngology patients for scheduled appointments","volume":"121","author":"Cohen","year":"2007","journal-title":"J Laryngol Otol"},{"issue":"2","key":"2020110612223593600_ocy002-B11","doi-asserted-by":"crossref","first-page":"259","DOI":"10.1016\/j.cgh.2015.07.051","article-title":"Factors associated with missed and cancelled colonoscopy appointments at veterans health administration facilities","volume":"14","author":"Partin","year":"2016","journal-title":"Clin Gastroenterol Hepatol"},{"key":"2020110612223593600_ocy002-B12","doi-asserted-by":"crossref","first-page":"428","DOI":"10.1016\/j.dss.2012.10.048","article-title":"An empirical investigation into factors affecting patient cancellations and no-shows at outpatient clinics","volume":"57","author":"Norris","year":"2014","journal-title":"Decis Support Syst"},{"key":"2020110612223593600_ocy002-B13","first-page":"1163","article-title":"Factors associated with nonattendance at clinical medicine scheduled outpatient appointments in a university general hospital","volume":"7","author":"Giunta","year":"2013","journal-title":"Patient Prefer Adherence"},{"issue":"1","key":"2020110612223593600_ocy002-B14","doi-asserted-by":"crossref","first-page":"15","DOI":"10.1007\/BF03262661","article-title":"Use of mobile telephone short message service (SMS) as a reminder: the effect on patient attendance","volume":"10","author":"Foley","year":"2009","journal-title":"Eur Arch Paediatr Dent"},{"issue":"4","key":"2020110612223593600_ocy002-B15","doi-asserted-by":"crossref","first-page":"355","DOI":"10.1007\/s11845-008-0187-2","article-title":"Potential impact of text message reminders on non-attendance at outpatient clinics","volume":"177","author":"Hogan","year":"2008","journal-title":"Ir J Med Sci"},{"issue":"5","key":"2020110612223593600_ocy002-B16","first-page":"278","article-title":"The effectiveness of reminder phone calls on reducing no-show rates in ambulatory care","volume":"29","author":"Woods","year":"2011","journal-title":"Nurs Econ"},{"key":"2020110612223593600_ocy002-B17","doi-asserted-by":"crossref","first-page":"79","DOI":"10.1186\/1471-2296-11-79","article-title":"Reduction of missed appointments at an urban primary care clinic: a randomised controlled study","volume":"11","author":"Perron","year":"2010","journal-title":"BMC Fam Pract"},{"issue":"1","key":"2020110612223593600_ocy002-B18","doi-asserted-by":"crossref","first-page":"267","DOI":"10.1111\/j.2517-6161.1996.tb02080.x","article-title":"Regression shrinkage and selection via the lasso","volume":"58","author":"Tibshirani","year":"1996","journal-title":"J R Stat Soc Ser B"},{"issue":"1","key":"2020110612223593600_ocy002-B19","doi-asserted-by":"crossref","first-page":"1","DOI":"10.18637\/jss.v033.i01","article-title":"Regularization paths for generalized linear models via coordinate descent","volume":"33","author":"Friedman","year":"2010","journal-title":"J Stat Softw"},{"issue":"4","key":"2020110612223593600_ocy002-B20","doi-asserted-by":"crossref","first-page":"1692","DOI":"10.1177\/0962280213497434","article-title":"Assessing calibration of prognostic risk scores","volume":"25","author":"Crowson","year":"2016","journal-title":"Stat Methods Med Res"},{"key":"2020110612223593600_ocy002-B21","article-title":"Does ignoring clustering in multicenter data influence the performance of prediction models? A simulation study","author":"Wynants","year":"2016","journal-title":"Stat Methods Med Res"},{"issue":"3","key":"2020110612223593600_ocy002-B22","doi-asserted-by":"crossref","first-page":"E1","DOI":"10.1177\/0272989X12439755","article-title":"Adaptation of clinical prediction models for application in local settings","volume":"32","author":"Kappen","year":"2012","journal-title":"Med Decis Mak Int J Soc Med Decis Mak"},{"issue":"12","key":"2020110612223593600_ocy002-B23","doi-asserted-by":"crossref","first-page":"1406","DOI":"10.1016\/j.jclinepi.2015.02.002","article-title":"A simulation study of sample size demonstrated the importance of the number of events per variable to develop prediction models in clustered data","volume":"68","author":"Wynants","year":"2015","journal-title":"J Clin Epidemiol"},{"key":"2020110612223593600_ocy002-B24","doi-asserted-by":"crossref","first-page":"19","DOI":"10.1186\/1471-2288-13-19","article-title":"Prediction models for clustered data: comparison of a random intercept and standard regression model","volume":"13","author":"Bouwmeester","year":"2013","journal-title":"BMC Med Res Methodol"},{"issue":"10","key":"2020110612223593600_ocy002-B25","doi-asserted-by":"crossref","first-page":"749","DOI":"10.1016\/j.annepidem.2005.03.001","article-title":"Public health application comparing multilevel analysis with logistic regression: immunization coverage among long-term care facility residents","volume":"15","author":"Bardenheier","year":"2005","journal-title":"Ann Epidemiol"},{"key":"2020110612223593600_ocy002-B26","doi-asserted-by":"crossref","first-page":"5","DOI":"10.1186\/1471-2288-14-5","article-title":"Assessing discriminative ability of risk models in clustered data","volume":"14","author":"van Klaveren","year":"2014","journal-title":"BMC Med Res Methodol"},{"key":"2020110612223593600_ocy002-B27","doi-asserted-by":"crossref","first-page":"418","DOI":"10.1016\/j.jbi.2014.08.006","article-title":"The effects of data sources, cohort selection, and outcome definition on a predictive model of risk of thirty-day hospital readmissions","volume":"52","author":"Walsh","year":"2014","journal-title":"J Biomed Inform"},{"key":"2020110612223593600_ocy002-B28","doi-asserted-by":"crossref","first-page":"e3230","DOI":"10.7717\/peerj.3230","article-title":"Contribution of temporal data to predictive performance in 30-day readmission of morbidly obese patients","volume":"5","author":"Povalej Brzan","year":"2017","journal-title":"Peer J"},{"issue":"5","key":"2020110612223593600_ocy002-B29","doi-asserted-by":"crossref","first-page":"655","DOI":"10.1007\/s11695-010-0325-6","article-title":"A nomogram for predicting surgical complications in bariatric surgery patients","volume":"21","author":"Turner","year":"2011","journal-title":"Obes Surg"},{"issue":"3","key":"2020110612223593600_ocy002-B30","doi-asserted-by":"crossref","first-page":"226","DOI":"10.1186\/cc11904","article-title":"Clinical review: can we predict which patients are at risk of complications following surgery?","volume":"17","author":"Shah","year":"2013","journal-title":"Crit Care"},{"issue":"1","key":"2020110612223593600_ocy002-B31","doi-asserted-by":"crossref","first-page":"10","DOI":"10.1002\/wics.1282","article-title":"Statistical learning methods for longitudinal high-dimensional data","volume":"6","author":"Chen","year":"2014","journal-title":"Wiley Interdiscip Rev Comput Stat"},{"key":"2020110612223593600_ocy002-B32","article-title":"Efficient model-based clustering with coalescents: application to multiple outcomes using medical records data","author":"Henao","year":"2016","journal-title":"ArXiv Prepr ArXiv160803191"}],"container-title":["Journal of the American Medical Informatics Association"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/academic.oup.com\/jamia\/article-pdf\/25\/8\/924\/34150438\/ocy002.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"http:\/\/academic.oup.com\/jamia\/article-pdf\/25\/8\/924\/34150438\/ocy002.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,7,2]],"date-time":"2024-07-02T23:24:35Z","timestamp":1719962675000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/jamia\/article\/25\/8\/924\/4849782"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2018,2,9]]},"references-count":32,"journal-issue":{"issue":"8","published-online":{"date-parts":[[2018,2,9]]},"published-print":{"date-parts":[[2018,8,1]]}},"URL":"https:\/\/doi.org\/10.1093\/jamia\/ocy002","relation":{},"ISSN":["1067-5027","1527-974X"],"issn-type":[{"value":"1067-5027","type":"print"},{"value":"1527-974X","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2018,8]]},"published":{"date-parts":[[2018,2,9]]}}}