{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,14]],"date-time":"2026-07-14T11:18:06Z","timestamp":1784027886143,"version":"3.55.0"},"reference-count":33,"publisher":"Oxford University Press (OUP)","issue":"11","license":[{"start":{"date-parts":[[2022,8,26]],"date-time":"2022-08-26T00:00:00Z","timestamp":1661472000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/academic.oup.com\/journals\/pages\/open_access\/funder_policies\/chorus\/standard_publication_model"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2022,10,7]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:sec>\n                  <jats:title>Objective<\/jats:title>\n                  <jats:p>To assess the efficacy of interruptive electronic alerts in improving adherence to the American Board of Internal Medicine\u2019s Choosing Wisely recommendations to reduce unnecessary laboratory testing.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Materials and methods<\/jats:title>\n                  <jats:p>We administered 5 cluster randomized controlled trials simultaneously, using electronic medical record alerts regarding prostate-specific antigen (PSA) testing, acute sinusitis treatment, vitamin D testing, carotid artery ultrasound screening, and human papillomavirus testing. For each alert, we assigned 5 outpatient clinics to an interruptive alert and 5 were observed as a control. Primary and secondary outcomes were the number of postalert orders per 100 patients at each clinic and number of triggered alerts divided by orders, respectively. Post hoc analysis evaluated whether physicians experiencing interruptive alerts reduced their alert-triggering behaviors.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Results<\/jats:title>\n                  <jats:p>Median postalert orders per 100 patients did not differ significantly between treatment and control groups; absolute median differences ranging from 0.04 to 0.40 for PSA testing. Median alerts per 100 orders did not differ significantly between treatment and control groups; absolute median differences ranged from 0.004 to 0.03. In post hoc analysis, providers receiving alerts regarding PSA testing in men were significantly less likely to trigger additional PSA alerts than those in the control sites (Incidence Rate Ratio 0.12, 95% CI [0.03\u20130.52]).<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Discussion<\/jats:title>\n                  <jats:p>Interruptive point-of-care alerts did not yield detectable changes in the overall rate of undesired orders or the order-to-alert ratio between active and silent sites. Complementary behavioral or educational interventions are likely needed to improve efforts to curb medical overuse.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Conclusion<\/jats:title>\n                  <jats:p>Implementation of interruptive alerts at the time of ordering was not associated with improved adherence to 5 Choosing Wisely guidelines.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Trial Registration<\/jats:title>\n                  <jats:p>NCT02709772<\/jats:p>\n               <\/jats:sec>","DOI":"10.1093\/jamia\/ocac139","type":"journal-article","created":{"date-parts":[[2022,8,26]],"date-time":"2022-08-26T16:08:27Z","timestamp":1661530107000},"page":"1941-1948","source":"Crossref","is-referenced-by-count":5,"title":["Interruptive Electronic Alerts for Choosing Wisely Recommendations: A Cluster Randomized Controlled Trial"],"prefix":"10.1093","volume":"29","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-5570-0536","authenticated-orcid":false,"given":"Vy T","family":"Ho","sequence":"first","affiliation":[{"name":"Division of Vascular Surgery, Department of Surgery, Stanford University School of Medicine , Stanford, California, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Rachael C","family":"Aikens","sequence":"additional","affiliation":[{"name":"Department of Biomedical Informatics, Stanford University School of Medicine , Stanford, California, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Geoffrey","family":"Tso","sequence":"additional","affiliation":[{"name":"Division of Primary Care and Population Health, Stanford University School of Medicine , Palo Alto, California, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Paul A","family":"Heidenreich","sequence":"additional","affiliation":[{"name":"Division of Cardiovascular Medicine, Department of Medicine, Stanford University School of Medicine , Stanford, California, USA"},{"name":"Center for Innovation to Implementation, VA Palo Alto Healthcare System , Palo Alto, California, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Christopher","family":"Sharp","sequence":"additional","affiliation":[{"name":"Department of Medicine, Stanford University School of Medicine , Stanford, California, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Steven M","family":"Asch","sequence":"additional","affiliation":[{"name":"Division of Primary Care and Population Health, Stanford University School of Medicine , Palo Alto, California, USA"},{"name":"Center for Innovation to Implementation, VA Palo Alto Healthcare System , Palo Alto, California, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jonathan H","family":"Chen","sequence":"additional","affiliation":[{"name":"Stanford Center for Biomedical Informatics Research, Department of Medicine, Stanford University , Stanford, California, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Neil K","family":"Shah","sequence":"additional","affiliation":[{"name":"Department of Pathology, Stanford University School of Medicine , Stanford, California, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"286","published-online":{"date-parts":[[2022,8,26]]},"reference":[{"issue":"12","key":"2023050511264404200_ocac139-B1","doi-asserted-by":"crossref","first-page":"1743","DOI":"10.2105\/AJPH.2020.305865","article-title":"Excess medical care spending: the categories, magnitude, and opportunity costs of wasteful spending in the United States","volume":"110","author":"Speer","year":"2020","journal-title":"Am J Public Health"},{"key":"2023050511264404200_ocac139-B2","author":"Choosing Wisely","year":"2014"},{"issue":"5","key":"2023050511264404200_ocac139-B3","doi-asserted-by":"crossref","first-page":"355","DOI":"10.1136\/bmjqs-2017-006699","article-title":"Precommitting to choose wisely about low-value services: a stepped wedge cluster randomised trial","volume":"27","author":"Kullgren","year":"2018","journal-title":"BMJ Qual Saf"},{"issue":"9","key":"2023050511264404200_ocac139-B4","doi-asserted-by":"crossref","first-page":"1135","DOI":"10.1016\/j.jacc.2017.06.065","article-title":"Improving the appropriate use of transthoracic echocardiography: the echo WISELY trial","volume":"70","author":"Bhatia","year":"2017","journal-title":"J Am Coll Cardiol"},{"key":"2023050511264404200_ocac139-B5","first-page":"1024","article-title":"The impact of choosing wisely interventions on low-value medical services: a systematic review","author":"Cliff","journal-title":"Milbank Q"},{"key":"2023050511264404200_ocac139-B6","doi-asserted-by":"crossref","first-page":"103956","DOI":"10.1016\/j.ijmedinf.2019.103956","article-title":"Interrupting providers with clinical decision support to improve care for heart failure","volume":"131","author":"Blecker","year":"2019","journal-title":"Int J Med Inf"},{"issue":"2","key":"2023050511264404200_ocac139-B7","doi-asserted-by":"crossref","first-page":"e12469","DOI":"10.2196\/12469","article-title":"Interruptive versus noninterruptive clinical decision support: usability study","volume":"6","author":"Blecker","year":"2019","journal-title":"JMIR Hum Factors"},{"issue":"2","key":"2023050511264404200_ocac139-B8","doi-asserted-by":"crossref","first-page":"121","DOI":"10.1080\/00223890802634175","article-title":"The analysis of count data: a gentle introduction to Poisson regression and its alternatives","volume":"91","author":"Coxe","year":"2009","journal-title":"J Pers Assess"},{"key":"2023050511264404200_ocac139-B9","volume-title":"Logistic Regression Using SAS: Theory and Application","author":"Allison","year":"2012","edition":"2nd ed."},{"key":"2023050511264404200_ocac139-B10","volume-title":"Python 3 Reference Manual","author":"Van Rossum","year":"2009"},{"key":"2023050511264404200_ocac139-B11","volume-title":"R: A Language and Environment for Statistical Computing","author":"R, Core Team","year":"2021"},{"key":"2023050511264404200_ocac139-B12","doi-asserted-by":"crossref","first-page":"1","DOI":"10.18637\/jss.v067.i01","article-title":"Fitting linear mixed-effects models using lme4","volume":"67","author":"Bates","year":"2015","journal-title":"J Stat Softw"},{"issue":"8","key":"2023050511264404200_ocac139-B13","doi-asserted-by":"crossref","first-page":"1133","DOI":"10.1007\/s11606-015-3249-y","article-title":"Reducing PSA-based prostate cancer screening in men aged\u200975 years and older with the use of highly specific computerized clinical decision support","volume":"30","author":"Shelton","year":"2015","journal-title":"J Gen Intern Med"},{"issue":"1","key":"2023050511264404200_ocac139-B14","doi-asserted-by":"crossref","first-page":"160","DOI":"10.1055\/s-0040-1701678","article-title":"Is the climb worth the view? The savings\/alert ratio for reducing vitamin D testing","volume":"11","author":"Hendrickson","year":"2020","journal-title":"Appl Clin Inform"},{"issue":"4","key":"2023050511264404200_ocac139-B15","doi-asserted-by":"crossref","first-page":"776","DOI":"10.1093\/jamia\/ocw182","article-title":"Decrease in unnecessary vitamin D testing using clinical decision support tools: making it harder to do the wrong thing","volume":"24","author":"Felcher","year":"2017","journal-title":"J Am Med Inform Assoc"},{"issue":"11","key":"2023050511264404200_ocac139-B16","doi-asserted-by":"crossref","first-page":"1556","DOI":"10.1093\/jamia\/ocy112","article-title":"Efficacy and unintended consequences of hard-stop alerts in electronic health record systems: a systematic review","volume":"25","author":"Powers","year":"2018","journal-title":"J Am Med Inform Assoc"},{"issue":"9","key":"2023050511264404200_ocac139-B17","doi-asserted-by":"crossref","first-page":"e2015250","DOI":"10.1001\/jamanetworkopen.2020.15250","article-title":"Trends in low-value carotid imaging in the Veterans Health Administration from 2007 to 2016","volume":"3","author":"Anderson","year":"2020","journal-title":"JAMA Netw Open"},{"issue":"12","key":"2023050511264404200_ocac139-B18","doi-asserted-by":"crossref","first-page":"1913","DOI":"10.1001\/jamainternmed.2015.5441","article-title":"Early trends among seven recommendations from the choosing wisely campaign","volume":"175","author":"Rosenberg","year":"2015","journal-title":"JAMA Intern Med"},{"issue":"12","key":"2023050511264404200_ocac139-B19","doi-asserted-by":"crossref","first-page":"1409","DOI":"10.1001\/archinte.163.12.1409","article-title":"Effects of computerized physician order entry and clinical decision support systems on medication safety: a systematic review","volume":"163","author":"Kaushal","year":"2003","journal-title":"Arch Intern Med"},{"issue":"1","key":"2023050511264404200_ocac139-B20","doi-asserted-by":"crossref","first-page":"2","DOI":"10.1186\/s13012-019-0960-9","article-title":"Unpacking the complexities of de-implementing inappropriate health interventions","volume":"15","author":"Norton","year":"2020","journal-title":"Implement Sci"},{"issue":"6","key":"2023050511264404200_ocac139-B21","doi-asserted-by":"crossref","first-page":"1020","DOI":"10.1097\/SLA.0000000000003792","article-title":"Too much surgery: overcoming barriers to de-implementation of low-value surgery","volume":"271","author":"Berlin","year":"2020","journal-title":"Ann Surg"},{"issue":"4","key":"2023050511264404200_ocac139-B22","doi-asserted-by":"crossref","first-page":"400","DOI":"10.1197\/jamia.M2238","article-title":"Evaluation of outpatient computerized physician medication order entry systems: a systematic review","volume":"14","author":"Eslami","year":"2007","journal-title":"J Am Med Inform Assoc"},{"key":"2023050511264404200_ocac139-B23","doi-asserted-by":"crossref","first-page":"104673","DOI":"10.1016\/j.ijmedinf.2021.104673","article-title":"A systematic review of drug allergy alert systems","volume":"159","author":"Luri","year":"2022","journal-title":"Int J Med Inform"},{"issue":"2","key":"2023050511264404200_ocac139-B24","doi-asserted-by":"crossref","first-page":"409","DOI":"10.1093\/jamia\/ocw072","article-title":"Alert override as a habitual behavior \u2013 a new perspective on a persistent problem","volume":"24","author":"Baysari","year":"2017","journal-title":"J Am Med Inform Assoc"},{"key":"2023050511264404200_ocac139-B25","doi-asserted-by":"crossref","first-page":"22","DOI":"10.1016\/j.ijmedinf.2017.05.011","article-title":"A systematic review of the effectiveness of interruptive medication prescribing alerts in hospital CPOE systems to change prescriber behavior and improve patient safety","volume":"105","author":"Page","year":"2017","journal-title":"Int J Med Inform"},{"issue":"2","key":"2023050511264404200_ocac139-B26","doi-asserted-by":"crossref","first-page":"ooab031","DOI":"10.1093\/jamiaopen\/ooab031","article-title":"A thematic analysis to examine the feasibility of EHR-based clinical decision support for implementing Choosing Wisely\u00ae guidelines","volume":"4","author":"Douthit","year":"2021","journal-title":"JAMIA Open"},{"issue":"1","key":"2023050511264404200_ocac139-B27","doi-asserted-by":"crossref","first-page":"36","DOI":"10.1186\/s12911-017-0430-8","article-title":"Effects of workload, work complexity, and repeated alerts on alert fatigue in a clinical decision support system","volume":"17","author":"Ancker","year":"2017","journal-title":"BMC Med Inform Decis Mak"},{"issue":"2","key":"2023050511264404200_ocac139-B28","doi-asserted-by":"crossref","first-page":"e23355","DOI":"10.2196\/23355","article-title":"Habit and automaticity in medical alert override: cohort study","volume":"24","author":"Wang","year":"2022","journal-title":"J Med Internet Res"},{"issue":"4","key":"2023050511264404200_ocac139-B29","doi-asserted-by":"crossref","first-page":"e000674","DOI":"10.1136\/bmjoq-2019-000674","article-title":"Reducing waste: a guidelines-based approach to reducing inappropriate vitamin D and TSH testing in the inpatient rehabilitation setting","volume":"8","author":"Bateman","year":"2019","journal-title":"BMJ Open Qual"},{"issue":"6","key":"2023050511264404200_ocac139-B30","doi-asserted-by":"crossref","first-page":"392","DOI":"10.12788\/jhm.2892","article-title":"The use of clinical decision support in reducing diagnosis of and treatment of asymptomatic bacteriuria","volume":"13","author":"Keller","year":"2018","journal-title":"J Hosp Med"},{"issue":"6","key":"2023050511264404200_ocac139-B31","doi-asserted-by":"crossref","first-page":"562","DOI":"10.1001\/jama.2016.0275","article-title":"Effect of behavioral interventions on inappropriate antibiotic prescribing among primary care practices: a randomized clinical trial","volume":"315","author":"Meeker","year":"2016","journal-title":"JAMA"},{"issue":"6","key":"2023050511264404200_ocac139-B32","doi-asserted-by":"crossref","first-page":"890","DOI":"10.1001\/jamainternmed.2014.949","article-title":"Reduction of inappropriate benzodiazepine prescriptions among older adults through direct patient education: the EMPOWER cluster randomized trial","volume":"174","author":"Tannenbaum","year":"2014","journal-title":"JAMA Intern Med"},{"issue":"1","key":"2023050511264404200_ocac139-B33","doi-asserted-by":"crossref","first-page":"e000015","DOI":"10.1136\/bmjhci-2019-000015","article-title":"Clinician preferences for computerised clinical decision support for medications in primary care: a focus group study","volume":"26","author":"Trinkley","year":"2019","journal-title":"BMJ Health Care Inform"}],"container-title":["Journal of the American Medical Informatics Association"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/jamia\/article-pdf\/29\/11\/1941\/50211774\/ocac139.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/jamia\/article-pdf\/29\/11\/1941\/50211774\/ocac139.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,5,5]],"date-time":"2023-05-05T11:27:12Z","timestamp":1683286032000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/jamia\/article\/29\/11\/1941\/6677250"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,8,26]]},"references-count":33,"journal-issue":{"issue":"11","published-online":{"date-parts":[[2022,8,26]]},"published-print":{"date-parts":[[2022,10,7]]}},"URL":"https:\/\/doi.org\/10.1093\/jamia\/ocac139","relation":{},"ISSN":["1067-5027","1527-974X"],"issn-type":[{"value":"1067-5027","type":"print"},{"value":"1527-974X","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2022,11,1]]},"published":{"date-parts":[[2022,8,26]]}}}