{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,17]],"date-time":"2026-08-17T23:27:01Z","timestamp":1787009221512,"version":"build-2736575974"},"reference-count":39,"publisher":"Oxford University Press (OUP)","issue":"7-8","license":[{"start":{"date-parts":[[2020,6,27]],"date-time":"2020-06-27T00:00:00Z","timestamp":1593216000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/academic.oup.com\/journals\/pages\/open_access\/funder_policies\/chorus\/standard_publication_model"}],"funder":[{"DOI":"10.13039\/100000738","name":"Department of Veterans Affairs","doi-asserted-by":"publisher","id":[{"id":"10.13039\/100000738","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2020,8,14]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:sec>\n                    <jats:title>Introduction<\/jats:title>\n                    <jats:p>No-shows are detrimental to both patients\u2019 health and health care systems. Literature documents no-show rates ranging from 10% in primary care clinics to over 60% in mental health clinics. Our model predicts the probability that a mental health clinic outpatient appointment will not be completed and identifies actionable variables associated with lowering the probability of no-show.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Materials and Methods<\/jats:title>\n                    <jats:p>We were granted access to de-identified administrative data from the Veterans Administration Corporate Data Warehouse related to appointments at 13 Veterans Administration Medical Centers. Our modeling data set included 1,206,271 unique appointment records scheduled to occur between January 1, 2013 and February 28, 2017. The training set included 846,668 appointment records scheduled between January 1, 2013 and December 31, 2015. The testing set included 359,603 appointment records scheduled between January 1, 2016 and February 28, 2017. The dependent binary variable was whether the appointment was completed or not. Independent variables were categorized into seven clusters: patient\u2019s demographics, appointment characteristics, patient\u2019s attendance history, alcohol use screening score, medications and medication possession ratios, prior diagnoses, and past utilization of Veterans Health Administration services. We used a forward stepwise selection, based on the likelihood ratio, to choose the variables in the model. The predictive model was built using the SAS HPLOGISTIC procedure.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>The best indicator of whether someone will miss an appointment is their historical attendance behavior. The top three variables associated with higher probabilities of a no-show were: the no-show rate over the previous 2 years before the current appointment, the no-show probability derived from the Markov model, and the age of the appointment. The top three variables that decrease the chance of no-showing were: the appointment was a new consult, the appointment was an overbook, and the patient had multiple appointments on the same day. The average of the areas under the receiver operating characteristic curves was 0.7577 for the training dataset, and 0.7513 for the test set.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusions<\/jats:title>\n                    <jats:p>The National Initiative to Reduce Missed Opportunities-2 confirmed findings that previous patient attendance is one of the key predictors of a future attendance and provides an additional layer of complexity for analyzing the effect of a patient\u2019s past behavior on future attendance. The National Initiative to Reduce Missed Opportunities-2 establishes that appointment attendance is related to medication adherence, particularly for medications used for treatment of mood disorders or to block the effects of opioids. However, there is no way to confirm whether a patient is actually taking medications as prescribed. Thus, a low medication possession ratio is an informative, albeit not a perfect, measure. It is our intention to further explore how diagnosis and medications can be better captured and used in predictive modeling of no-shows. Our findings on the effects of different factors on no-show rates can be used to predict individual no-show probabilities, and to identify patients who are high risk for missing appointments. The ability to predict a patient\u2019s risk of missing an appointment would allow for both advanced interventions to decrease no-shows and for more efficient scheduling.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.1093\/milmed\/usaa095","type":"journal-article","created":{"date-parts":[[2020,6,27]],"date-time":"2020-06-27T04:24:26Z","timestamp":1593231866000},"page":"e988-e994","source":"Crossref","is-referenced-by-count":21,"title":["Modeling Patient No-Show History and Predicting Future Appointment Behavior at the Veterans Administration\u2019s Outpatient Mental Health Clinics: NIRMO-2"],"prefix":"10.1093","volume":"185","author":[{"given":"Aleksandra Sasha","family":"Milicevic","sequence":"first","affiliation":[{"name":"Joseph M. Katz Graduate School of Business, University of Pittsburgh, 2809 Posvar Hall 230 S Bouquet St, Pittsburgh, PA 15213"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Kannop","family":"Mitsantisuk","sequence":"additional","affiliation":[{"name":"Joseph M. Katz Graduate School of Business, University of Pittsburgh, 2809 Posvar Hall 230 S Bouquet St, Pittsburgh, PA 15213"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Andrew","family":"Tjader","sequence":"additional","affiliation":[{"name":"Joseph M. Katz Graduate School of Business, University of Pittsburgh, 2809 Posvar Hall 230 S Bouquet St, Pittsburgh, PA 15213"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Dominic L","family":"Vargas","sequence":"additional","affiliation":[{"name":"Joseph M. Katz Graduate School of Business, University of Pittsburgh, 2809 Posvar Hall 230 S Bouquet St, Pittsburgh, PA 15213"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Terrence L","family":"Hubert","sequence":"additional","affiliation":[{"name":"Office of Strategic Integration, Veterans Engineering Resource Center, 1010 Delafield Road, 001VERC-A, Bldg. 70, Room BA014, Pittsburgh, PA 15215"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Brianna","family":"Scott","sequence":"additional","affiliation":[{"name":"VA Pittsburgh Healthcare System, 1010 Delafield Road, 001VERC-A, Bldg. 70, Room BA014, Pittsburgh, PA 15215"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"286","published-online":{"date-parts":[[2020,6,27]]},"reference":[{"key":"2021062700030356900_ref1","doi-asserted-by":"crossref","first-page":"800","DOI":"10.1080\/07408170802165880","article-title":"Appointment scheduling in health care: challenges and opportunities","volume":"40","author":"Gupta","year":"2008","journal-title":"IIE Trans"},{"key":"2021062700030356900_ref2","doi-asserted-by":"crossref","first-page":"412","DOI":"10.1016\/j.healthpol.2018.02.002","article-title":"No-shows in appointment scheduling\u2013a systematic literature review","volume":"122","author":"Dantas","year":"2018","journal-title":"Health Policy"},{"key":"2021062700030356900_ref3","first-page":"112","article-title":"Large-scale assessment of missed opportunity risks in a complex hospital setting","volume":"41","author":"Peng","year":"2016","journal-title":"Inform Health Soc Care"},{"key":"2021062700030356900_ref4","doi-asserted-by":"crossref","first-page":"e1708","DOI":"10.7205\/MILMED-D-16-00345","article-title":"Modeling patient no-show history and predicting future outpatient appointment behavior in the veterans health administration","volume":"182","author":"Goffman","year":"2017","journal-title":"Mil Med"},{"key":"2021062700030356900_ref5","doi-asserted-by":"crossref","first-page":"13","DOI":"10.1186\/s12913-015-1243-z","article-title":"Prevalence, predictors and economic consequences of no-shows","volume":"16","author":"Kheirkhah","year":"2016","journal-title":"BMC Health Ser Res"},{"key":"2021062700030356900_ref6","doi-asserted-by":"crossref","first-page":"79","DOI":"10.1186\/1471-2296-11-79","article-title":"Reduction of missed appointments at an urban primary care clinic: a randomised controlled study","volume":"11","author":"Perron","year":"2010","journal-title":"BMC Fam Pract"},{"key":"2021062700030356900_ref7","doi-asserted-by":"crossref","first-page":"689","DOI":"10.1097\/MLR.0000000000000543","article-title":"Reducing missed primary care appointments in a learning health system","volume":"54","author":"Steiner","year":"2016","journal-title":"Med Care"},{"key":"2021062700030356900_ref8","first-page":"627","article-title":"Failure to keep clinic appointments: implications for residency education and productivity","volume":"31","author":"Hixon","year":"1999","journal-title":"Fam Med"},{"key":"2021062700030356900_ref9","doi-asserted-by":"crossref","first-page":"178","DOI":"10.1093\/fampra\/20.2.178","article-title":"Non-attendance in general practice: a systematic review and its implications for access to primary health care","volume":"20","author":"George","year":"2003","journal-title":"Fam Pract"},{"key":"2021062700030356900_ref10","doi-asserted-by":"crossref","first-page":"531","DOI":"10.1176\/ps.2006.57.4.531","article-title":"Predictors of missed first appointments at community mental health centers after psychiatric hospitalization","volume":"57","author":"Compton","year":"2006","journal-title":"Psychiatr Ser"},{"key":"2021062700030356900_ref11","volume-title":"Audit of Veterans Health Administration's Efforts to Reduce Unused Outpatient Appointments","author":"Department of Veterans Affairs Office of Inspector General","year":"2008"},{"key":"2021062700030356900_ref12","volume-title":"Evidence Brief: Comparative Effectiveness of Appointment Recall Reminder Procedures for Follow-up Appointments","author":"Peterson","year":"2015"},{"key":"2021062700030356900_ref13","doi-asserted-by":"crossref","first-page":"1450","DOI":"10.1002\/pon.4570","article-title":"Trends in incidence and associated risk factors of suicide mortality among breast cancer patients","volume":"27","author":"Gaitanidis","year":"2018","journal-title":"Psycho-oncology"},{"key":"2021062700030356900_ref14","doi-asserted-by":"crossref","first-page":"205","DOI":"10.1007\/BF00757138","article-title":"Interventions for reducing missed initial appointments at a community mental health center","volume":"24","author":"Swenson","year":"1988","journal-title":"Community Ment Health J"},{"key":"2021062700030356900_ref15","doi-asserted-by":"crossref","first-page":"342","DOI":"10.1177\/070674370304800511","article-title":"Time of day influences nonattendance at urgent short-term mental health unit in Victoria, British Columbia","volume":"48","author":"Weinerman","year":"2003","journal-title":"Can J Psychiatry"},{"key":"2021062700030356900_ref16","doi-asserted-by":"crossref","first-page":"pii: E18","DOI":"10.3390\/healthcare7010018","article-title":"Rates and associated factors of secondary mental health care utilisation among Ex-Military personnel in the United States: a narrative review","volume":"7","author":"Mark","year":"2019","journal-title":"Healthcare"},{"key":"2021062700030356900_ref17","doi-asserted-by":"crossref","first-page":"546","DOI":"10.1016\/j.genhosppsych.2008.09.004","article-title":"Improving efficiency and access to mental health care: combining integrated care and advanced access","volume":"30","author":"Pomerantz","year":"2008","journal-title":"Gen Hosp Psychiatry"},{"key":"2021062700030356900_ref18","doi-asserted-by":"crossref","first-page":"5","DOI":"10.1002\/jts.20493","article-title":"VA mental health services utilization in Iraq and Afghanistan veterans in the first year of receiving new mental health diagnoses","volume":"23","author":"Seal","year":"2010","journal-title":"J Trauma Stress"},{"key":"2021062700030356900_ref19","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1037\/a0022705","article-title":"Pretreatment predictors of dropout from cognitive behavioral therapy for PTSD in Iraq and Afghanistan war veterans","volume":"8","author":"Garcia","year":"2011","journal-title":"Psychol Serv"},{"key":"2021062700030356900_ref20","volume-title":"Data Science for Business: What You Need to Know About Data Mining and Data-Analytic Thinking","author":"Provost","year":"2013"},{"key":"2021062700030356900_ref21","author":"Turkcan"},{"key":"2021062700030356900_ref22","first-page":"65","article-title":"Data mining applications in healthcar","volume":"19","author":"Koh","year":"2011","journal-title":"J Health Inf Manage"},{"key":"2021062700030356900_ref23","doi-asserted-by":"crossref","first-page":"155","DOI":"10.5121\/ijcsit.2011.3413","article-title":"Performance analysis of various data mining classification techniques on healthcare data","volume":"3","author":"Gupta","year":"2011","journal-title":"IJCSIT"},{"key":"2021062700030356900_ref24","doi-asserted-by":"crossref","first-page":"100","DOI":"10.1016\/j.imu.2017.12.006","article-title":"Type 2 diabetes mellitus prediction model based on data mining","volume":"10","author":"Wu","year":"2018","journal-title":"Inform Med Unlocked"},{"key":"2021062700030356900_ref25","doi-asserted-by":"crossref","first-page":"306","DOI":"10.1016\/j.procs.2015.12.145","article-title":"Data mining in healthcare\u2013a review","volume":"72","author":"Jothi","year":"2015","journal-title":"Procedia Comput Sci"},{"key":"2021062700030356900_ref26","doi-asserted-by":"crossref","first-page":"241","DOI":"10.14257\/ijbsbt.2013.5.5.25","article-title":"A survey on data mining approaches for healthcare","volume":"5","author":"Tomar","year":"2013","journal-title":"Int J Bio-Sci Bio-Technol"},{"key":"2021062700030356900_ref27","doi-asserted-by":"crossref","first-page":"519","DOI":"10.1111\/j.1937-5956.2003.tb00218.x","article-title":"Outpatient scheduling in health care: a review of literature","volume":"12","author":"Cayirli","year":"2003","journal-title":"Prod Oper Manag"},{"key":"2021062700030356900_ref28","doi-asserted-by":"crossref","first-page":"1351","DOI":"10.1001\/jama.2013.393","article-title":"The inevitable application of big data to health care","volume":"309","author":"Murdoch","year":"2013","journal-title":"JAMA"},{"key":"2021062700030356900_ref29","first-page":"597","article-title":"Implementation of evidence-based alcohol screening in the veterans health administration","volume":"12","author":"Bradley","year":"2006","journal-title":"Am J Manag Care"},{"key":"2021062700030356900_ref30","doi-asserted-by":"crossref","first-page":"1618","DOI":"10.1016\/j.socscimed.2005.03.017","article-title":"Relationship of anxiety, depression and alcohol use disorders to persistent high utilization and potentially problematic under-utilization of primary medical care","volume":"61","author":"Ford","year":"2005","journal-title":"Soc Sci Med"},{"key":"2021062700030356900_ref31","doi-asserted-by":"crossref","first-page":"448","DOI":"10.1097\/01.ALC.0000117836.38108.38","article-title":"Using alcohol screening results and treatment history to assess the severity of at-risk drinking in veterans affairs primary care patients","volume":"28","author":"Bradley","year":"2004","journal-title":"Alcohol Clin Exp Res"},{"key":"2021062700030356900_ref32","doi-asserted-by":"crossref","first-page":"10","DOI":"10.1016\/j.jsat.2014.12.003","article-title":"National prevalence and correlates of alcohol misuse in women veterans","volume":"52","author":"Hoggatt","year":"2015","journal-title":"J Subst Abuse Treat"},{"key":"2021062700030356900_ref33","doi-asserted-by":"crossref","first-page":"487","DOI":"10.1056\/NEJMra050100","article-title":"Adherence to medication","volume":"353","author":"Osterberg","year":"2005","journal-title":"N Engl J Med"},{"key":"2021062700030356900_ref34","doi-asserted-by":"crossref","first-page":"255","DOI":"10.1093\/oxfordjournals.schbul.a007076","article-title":"Poor antipsychotic adherence among patients with schizophrenia: medication and patient factors","volume":"30","author":"Valenstein","year":"2004","journal-title":"Schizophr Bull"},{"key":"2021062700030356900_ref35","doi-asserted-by":"crossref","first-page":"1190","DOI":"10.1097\/01.ALC.0000171937.87731.28","article-title":"A temporal and dose-response association between alcohol consumption and medication adherence among veterans in care","volume":"29","author":"Braithwaite","year":"2005","journal-title":"Alcohol Clin Exp Res"},{"key":"2021062700030356900_ref36","doi-asserted-by":"crossref","first-page":"26","DOI":"10.1016\/j.amjmed.2006.06.028","article-title":"Antihypertensive medication adherence in the Department of Veterans Affairs","volume":"120","author":"Siegel","year":"2007","journal-title":"Am J Med"},{"key":"2021062700030356900_ref37","author":"VSSC No Shows and Cancellations Cube for Fiscal Year 2018"},{"key":"2021062700030356900_ref38","doi-asserted-by":"crossref","first-page":"e0214869","DOI":"10.1371\/journal.pone.0214869","article-title":"Development and validation of a patient no-show predictive model at a primary care setting in southern Brazil","volume":"14","author":"Lenzi","year":"2019","journal-title":"PloS One"},{"key":"2021062700030356900_ref39","first-page":"13","article-title":"Can what we learned about reducing no-shows in our clinic work for you","volume":"13","author":"Gajwani","year":"2014","journal-title":"Curr Psychiatry"}],"container-title":["Military Medicine"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/academic.oup.com\/milmed\/article-pdf\/185\/7-8\/e988\/33655597\/usaa095.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"http:\/\/academic.oup.com\/milmed\/article-pdf\/185\/7-8\/e988\/33655597\/usaa095.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,6,26]],"date-time":"2021-06-26T20:03:32Z","timestamp":1624737812000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/milmed\/article\/185\/7-8\/e988\/5863443"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,6,27]]},"references-count":39,"journal-issue":{"issue":"7-8","published-online":{"date-parts":[[2020,6,27]]},"published-print":{"date-parts":[[2020,8,14]]}},"URL":"https:\/\/doi.org\/10.1093\/milmed\/usaa095","relation":{},"ISSN":["0026-4075","1930-613X"],"issn-type":[{"value":"0026-4075","type":"print"},{"value":"1930-613X","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2020,7]]},"published":{"date-parts":[[2020,6,27]]}}}