{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,9,16]],"date-time":"2025-09-16T18:12:51Z","timestamp":1758046371781,"version":"3.44.0"},"reference-count":42,"publisher":"Georg Thieme Verlag KG","issue":"04","license":[{"start":{"date-parts":[[2025,8,20]],"date-time":"2025-08-20T00:00:00Z","timestamp":1755648000000},"content-version":"vor","delay-in-days":19,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/100000026","name":"National Institute on Drug Abuse","doi-asserted-by":"crossref","award":["R33DA059884"],"award-info":[{"award-number":["R33DA059884"]}],"id":[{"id":"10.13039\/100000026","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Appl Clin Inform"],"published-print":{"date-parts":[[2025,8]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>In the emergency department-initiated buprenorphine for opioid use disorder (EMBED) trial, a clinical decision support (CDS) tool had no effect on rates of buprenorphine initiation in emergency department (ED) patients with opioid use disorder. The Agency for Healthcare Research and Quality (AHRQ) recently released a CDS Performance Measure Inventory to guide data-driven CDS development and evaluation. Through partner co-design, we tailored AHRQ inventory measures to evaluate EMBED CDS performance and drive improvements.<\/jats:p><jats:p>Relevant AHRQ inventory measures were selected and adapted using a partner co-design approach grounded in consensus methodology, with three iterative, multidisciplinary partner working group sessions involving stakeholders from various roles and institutions; meetings were followed by postmeeting surveys. The co-design process was divided into conceptualization, specification, and evaluation phases building on the Centers for Medicare and Medicaid Services' measure life cycle framework. Final measures were evaluated in three EDs in a single health system from January 1, 2023, to December 31, 2024.<\/jats:p><jats:p>The partner working group included 25 members. During conceptualization, 13 initial candidate metrics were narrowed to 6 priority categories. These were further specified and validated as the following measures, presented with preliminary values based on the use of the current (i.e., preoptimization) EMBED CDS: eligible encounters with CDS engagement, 5.0% (95% confidence interval: 4.3\u20135.8%); teamwork on ED initiation of buprenorphine, 39.9% (32.5\u201347.3%); proportion of eligible users who used EMBED, 58.3% (50.9\u201365.8%); time spent on EMBED, 29.0\u2009seconds (20.4\u201337.7\u2009seconds); proportion of buprenorphine orders placed through EMBED, 6.5% (3.4\u20139.6%); and task completion, 13.8% (8.9\u201318.7%) for buprenorphine order\/prescription.<\/jats:p><jats:p>A measurement science framework informed by partner co-design was a feasible approach to develop measures to guide CDS improvement. Subsequent research could adapt this approach to evaluate other CDS applications.<\/jats:p>","DOI":"10.1055\/a-2595-0317","type":"journal-article","created":{"date-parts":[[2025,8,20]],"date-time":"2025-08-20T22:52:25Z","timestamp":1755730345000},"page":"1067-1076","source":"Crossref","is-referenced-by-count":0,"title":["A Measurement Science Framework to Optimize CDS for Opioid Use Disorder Treatment in the ED"],"prefix":"10.1055","volume":"16","author":[{"given":"Mark S.","family":"Iscoe","sequence":"additional","affiliation":[{"name":"Department of Emergency Medicine, Yale University School of Medicine, New Haven, Connecticut, United States"},{"name":"Department of Biomedical Informatics and Data Sciences, Yale University School of Medicine, New Haven, Connecticut, United States"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Carolina Diniz","family":"Hooper","sequence":"additional","affiliation":[{"name":"Department of Emergency Medicine, Yale University School of Medicine, New Haven, Connecticut, United States"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Deborah R.","family":"Levy","sequence":"additional","affiliation":[{"name":"Department of Biomedical Informatics and Data Sciences, Yale University School of Medicine, New Haven, Connecticut, United States"},{"name":"Department of Veterans Affairs, Amarillo VA Healthcare System, Amarillo, Texas, United States"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"John","family":"Lutz","sequence":"additional","affiliation":[{"name":"Yale University School of Medicine, New Haven, Connecticut, United States"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Hyung","family":"Paek","sequence":"additional","affiliation":[{"name":"Department of Biostatistics (Health Informatics), Yale School of Public Health, New Haven, Connecticut, United States"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Christian","family":"Rose","sequence":"additional","affiliation":[{"name":"Department of Emergency Medicine, Stanford University School of Medicine, Stanford, California, United States"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Thomas","family":"Kannampallil","sequence":"additional","affiliation":[{"name":"Institute for Informatics, Data Science, and Biostatistics Washington University School of Medicine, St. Louis, Missouri, United States"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Daniella","family":"Meeker","sequence":"additional","affiliation":[{"name":"Department of Biomedical Informatics and Data Sciences, Yale University School of Medicine, New Haven, Connecticut, United States"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"James D.","family":"Dziura","sequence":"additional","affiliation":[{"name":"Department of Emergency Medicine, Yale University School of Medicine, New Haven, Connecticut, United States"},{"name":"Department of Biostatistics (Health Informatics), Yale School of Public Health, New Haven, Connecticut, United States"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Edward R.","family":"Melnick","sequence":"additional","affiliation":[{"name":"Department of Emergency Medicine, Yale University School of Medicine, New Haven, Connecticut, United States"},{"name":"Department of Biomedical Informatics and Data Sciences, Yale University School of Medicine, New Haven, Connecticut, United States"},{"name":"Department of Biostatistics (Health Informatics), Yale School of Public Health, New Haven, Connecticut, United States"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"194","published-online":{"date-parts":[[2025,9,12]]},"reference":[{"issue":"01","key":"ref1","doi-asserted-by":"crossref","first-page":"19","DOI":"10.1016\/j.amepre.2023.01.037","article-title":"Buprenorphine after nonfatal opioid overdose: reduced mortality risk in medicare disability beneficiaries","volume":"65","author":"H Samples","year":"2023","journal-title":"Am J Prev Med"},{"key":"ref2","doi-asserted-by":"crossref","first-page":"111040","DOI":"10.1016\/j.drugalcdep.2023.111040","article-title":"Receipt of opioid use disorder treatments prior to fatal overdoses and comparison to no treatment in Connecticut, 2016-17","volume":"254","author":"R Heimer","year":"2024","journal-title":"Drug Alcohol Depend"},{"issue":"16","key":"ref3","doi-asserted-by":"crossref","first-page":"1402","DOI":"10.1001\/jama.2023.1207","article-title":"Trends in buprenorphine initiation and retention in the United States, 2016-2022","volume":"329","author":"K P Chua","year":"2023","journal-title":"JAMA"},{"issue":"03","key":"ref4","doi-asserted-by":"crossref","first-page":"137","DOI":"10.7326\/M17-3107","article-title":"Medication for opioid use disorder after nonfatal opioid overdose and association with mortality: a cohort study","volume":"169","author":"M R Larochelle","year":"2018","journal-title":"Ann Intern Med"},{"key":"ref5","doi-asserted-by":"crossref","first-page":"108568","DOI":"10.1016\/j.drugalcdep.2021.108568","article-title":"Prevalence and charges of opioid-related visits to U.S. emergency departments","volume":"221","author":"J R Langabeer","year":"2021","journal-title":"Drug Alcohol Depend"},{"issue":"16","key":"ref6","doi-asserted-by":"crossref","first-page":"1636","DOI":"10.1001\/jama.2015.3474","article-title":"Emergency department-initiated buprenorphine\/naloxone treatment for opioid dependence: a randomized clinical trial","volume":"313","author":"G D'Onofrio","year":"2015","journal-title":"JAMA"},{"issue":"01","key":"ref7","doi-asserted-by":"crossref","first-page":"e13121","DOI":"10.2196\/13121","article-title":"Computerized clinical decision support system for emergency department-initiated buprenorphine for opioid use disorder: user-centered design","volume":"6","author":"J M Ray","year":"2019","journal-title":"JMIR Hum Factors"},{"issue":"04","key":"ref8","doi-asserted-by":"crossref","first-page":"434","DOI":"10.1093\/jamiaopen\/ooz053","article-title":"An integrated web application for decision support and automation of EHR workflow: a case study of current challenges to standards-based messaging and scalability from the EMBED trial","volume":"2","author":"E R Melnick","year":"2019","journal-title":"JAMIA Open"},{"key":"ref9","doi-asserted-by":"crossref","first-page":"m3216","DOI":"10.1136\/bmj.m3216","article-title":"Computerised clinical decision support systems and absolute improvements in care: meta-analysis of controlled clinical trials","volume":"370","author":"J L Kwan","year":"2020","journal-title":"BMJ"},{"key":"ref10","doi-asserted-by":"crossref","first-page":"e069271","DOI":"10.1136\/bmj-2021-069271","article-title":"User centered clinical decision support to implement initiation of buprenorphine for opioid use disorder in the emergency department: EMBED pragmatic cluster randomized controlled trial","volume":"377","author":"E R Melnick","year":"2022","journal-title":"BMJ"},{"issue":"21","key":"ref11","doi-asserted-by":"crossref","first-page":"2149","DOI":"10.1001\/jama.2021.5487","article-title":"Implementing measurement science for electronic health record use","volume":"325","author":"E R Melnick","year":"2021","journal-title":"JAMA"},{"issue":"01","key":"ref12","doi-asserted-by":"crossref","first-page":"167","DOI":"10.1093\/jamia\/ocac173","article-title":"Using electronic health record audit log data for research: insights from early efforts","volume":"30","author":"T Kannampallil","year":"2022","journal-title":"J Am Med Inform Assoc"},{"issue":"03","key":"ref13","doi-asserted-by":"crossref","first-page":"784","DOI":"10.1093\/jamia\/ocad254","article-title":"Guidance for reporting analyses of metadata on electronic health record use","volume":"31","author":"A Rule","year":"2024","journal-title":"J Am Med Inform Assoc"},{"issue":"05","key":"ref14","doi-asserted-by":"crossref","first-page":"944","DOI":"10.1055\/a-2187-3243","article-title":"Identifying and addressing barriers to implementing core electronic health record use metrics for ambulatory care: virtual consensus conference proceedings","volume":"14","author":"D R Levy","year":"2023","journal-title":"Appl Clin Inform"},{"key":"ref15","doi-asserted-by":"crossref","first-page":"103343","DOI":"10.1016\/j.jbi.2019.103343","article-title":"EHR audit logs: a new goldmine for health services research?","volume":"101","author":"J Adler-Milstein","year":"2020","journal-title":"J Biomed Inform"},{"issue":"07","key":"ref17","doi-asserted-by":"crossref","first-page":"e014650","DOI":"10.1136\/bmjopen-2016-014650","article-title":"What outcomes are associated with developing and implementing co-produced interventions in acute healthcare settings? A rapid evidence synthesis","volume":"7","author":"D Clarke","year":"2017","journal-title":"BMJ Open"},{"issue":"06","key":"ref18","doi-asserted-by":"crossref","first-page":"1182","DOI":"10.3390\/ijerph15061182","article-title":"Using co-design to develop a collective leadership intervention for healthcare teams to improve safety culture","volume":"15","author":"M E Ward","year":"2018","journal-title":"Int J Environ Res Public Health"},{"issue":"03","key":"ref19","doi-asserted-by":"crossref","first-page":"235","DOI":"10.46747\/cfp.6803235","article-title":"Impact of patient partner co-design on survey development in primary care research: case study","volume":"68","author":"L Cooper","year":"2022","journal-title":"Can Fam Physician"},{"issue":"05","key":"ref20","doi-asserted-by":"crossref","first-page":"S112","DOI":"10.1016\/j.amepre.2007.01.022","article-title":"The multiphase optimization strategy (MOST) and the sequential multiple assignment randomized trial (SMART): new methods for more potent eHealth interventions","volume":"32","author":"L M Collins","year":"2007","journal-title":"Am J Prev Med"},{"issue":"02","key":"ref21","doi-asserted-by":"crossref","first-page":"e098072","DOI":"10.1136\/bmjopen-2024-098072","article-title":"Adaptive decision support for addiction treatment to implement initiation of buprenorphine for opioid use disorder in the emergency department: protocol for the ADAPT multiphase optimization strategy trial","volume":"15","author":"M S Iscoe","year":"2025","journal-title":"BMJ Open"},{"issue":"11","key":"ref23","doi-asserted-by":"crossref","first-page":"1873","DOI":"10.1080\/03007995.2020.1816946","article-title":"Delphi, non-RAND modified Delphi, RAND\/UCLA appropriateness method and a novel group awareness and consensus methodology for consensus measurement: a systematic literature review","volume":"36","author":"R Jandhyala","year":"2020","journal-title":"Curr Med Res Opin"},{"issue":"02","key":"ref24","doi-asserted-by":"crossref","first-page":"178","DOI":"10.1093\/eurjcn\/zvab127","article-title":"User experience (UX) design as a co-design methodology: lessons learned during the development of a web-based portal for cardiac rehabilitation","volume":"21","author":"K Nesbitt","year":"2022","journal-title":"Eur J Cardiovasc Nurs"},{"issue":"09","key":"ref25","first-page":"655","article-title":"Codesigning an E-health intervention for surgery preparation and recovery","volume":"42","author":"C Williams","year":"2024","journal-title":"Comput Inform Nurs"},{"issue":"15","key":"ref26","doi-asserted-by":"crossref","first-page":"1517","DOI":"10.1001\/jama.2014.12819","article-title":"What makes a good quality measure?","volume":"312","author":"E A McGlynn","year":"2014","journal-title":"JAMA"},{"issue":"04","key":"ref27","doi-asserted-by":"crossref","first-page":"e15794","DOI":"10.2196\/15794","article-title":"Identifying opioid use disorder in the emergency department: multi-system electronic health record-based computable phenotype derivation and validation study","volume":"7","author":"D Chartash","year":"2019","journal-title":"JMIR Med Inform"},{"issue":"05","key":"ref28","doi-asserted-by":"crossref","first-page":"1018","DOI":"10.1055\/s-0043-1777126","article-title":"A tool to identify and engage patients on risky opioid regimens","volume":"14","author":"A C Griffin","year":"2023","journal-title":"Appl Clin Inform"},{"issue":"04","key":"ref29","doi-asserted-by":"crossref","first-page":"705","DOI":"10.1055\/s-0043-1771395","article-title":"Behavioral health decision support systems and user interface design in the emergency department","volume":"14","author":"N W Jones","year":"2023","journal-title":"Appl Clin Inform"},{"issue":"03","key":"ref30","doi-asserted-by":"crossref","first-page":"602","DOI":"10.1055\/s-0042-1749332","article-title":"Evaluating a prototype clinical decision support tool for chronic pain treatment in primary care","volume":"13","author":"K S Allen","year":"2022","journal-title":"Appl Clin Inform"},{"issue":"04","key":"ref31","doi-asserted-by":"crossref","first-page":"820","DOI":"10.1055\/s-0042-1756366","article-title":"A qualitative description of clinician free-text rationales entered within accountable justification interventions","volume":"13","author":"T Brown","year":"2022","journal-title":"Appl Clin Inform"},{"key":"ref32","doi-asserted-by":"crossref","first-page":"100056","DOI":"10.1016\/j.hfh.2023.100056","article-title":"Post-implementation usability evaluation of a human factors-based clinical decision support for pulmonary embolism (PE) diagnosis (Dx): PE Dx Study Part 1","volume":"4","author":"M E Salwei","year":"2023","journal-title":"Hum Factors Healthc"},{"issue":"04","key":"ref33","doi-asserted-by":"crossref","first-page":"329","DOI":"10.1136\/bmjqs-2019-009857","article-title":"Application of human factors to improve usability of clinical decision support for diagnostic decision-making: a scenario-based simulation study","volume":"29","author":"P Carayon","year":"2020","journal-title":"BMJ Qual Saf"},{"key":"ref34","doi-asserted-by":"crossref","first-page":"e55316","DOI":"10.2196\/55316","article-title":"Snowball group usability testing for rapid and iterative multisite tool development: method development study","volume":"9","author":"K L Dauber-Decker","year":"2025","journal-title":"JMIR Form Res"},{"issue":"01","key":"ref35","doi-asserted-by":"crossref","first-page":"100598","DOI":"10.1016\/j.hjdsi.2021.100598","article-title":"Collaborative design and implementation of a clinical decision support system for automated fall-risk identification and referrals in emergency departments","volume":"10","author":"G C Jacobsohn","year":"2022","journal-title":"Healthcare (Amst)"},{"issue":"02","key":"ref36","doi-asserted-by":"crossref","first-page":"419","DOI":"10.1055\/s-0042-1745830","article-title":"Impact of a vendor-developed opioid clinical decision support intervention on adherence to prescribing guidelines, opioid prescribing, and rates of opioid-related encounters","volume":"13","author":"R P Pierce","year":"2022","journal-title":"Appl Clin Inform"},{"key":"ref37","doi-asserted-by":"crossref","first-page":"e58276","DOI":"10.2196\/58276","article-title":"Clinical decision support to increase emergency department naloxone coprescribing: implementation report","volume":"12","author":"S W Sommers","year":"2024","journal-title":"JMIR Med Inform"},{"issue":"01","key":"ref38","doi-asserted-by":"crossref","first-page":"41","DOI":"10.1186\/s43058-023-00421-7","article-title":"Implementation strategies to address the determinants of adoption, implementation, and maintenance of a clinical decision support tool for emergency department buprenorphine initiation: a qualitative study","volume":"4","author":"M J Simpson","year":"2023","journal-title":"Implement Sci Commun"},{"issue":"11","key":"ref39","doi-asserted-by":"crossref","first-page":"1556","DOI":"10.1093\/jamia\/ocy112","article-title":"Efficacy and unintended consequences of hard-stop alerts in electronic health record systems: a systematic review","volume":"25","author":"E M Powers","year":"2018","journal-title":"J Am Med Inform Assoc"},{"issue":"01","key":"ref40","doi-asserted-by":"crossref","first-page":"114","DOI":"10.1186\/s13012-018-0790-1","article-title":"A systematic review of trials evaluating success factors of interventions with computerised clinical decision support","volume":"13","author":"S Van de Velde","year":"2018","journal-title":"Implement Sci"},{"issue":"02","key":"ref41","doi-asserted-by":"crossref","first-page":"368","DOI":"10.4338\/ACI-2013-09-RA-0069","article-title":"Developing clinical decision support within a commercial electronic health record system to improve antimicrobial prescribing in the neonatal ICU","volume":"5","author":"R S Hum","year":"2014","journal-title":"Appl Clin Inform"},{"issue":"01","key":"ref42","doi-asserted-by":"crossref","first-page":"215","DOI":"10.1055\/a-2447-8463","article-title":"Iterative development of a clinical decision support tool to enhance naloxone coprescribing","volume":"16","author":"R Wu","year":"2025","journal-title":"Appl Clin Inform"},{"issue":"05","key":"ref43","doi-asserted-by":"crossref","first-page":"961","DOI":"10.1055\/s-0043-1776879","article-title":"Opioid prescribing for chronic musculoskeletal conditions: trends over time and implementation of safe opioid-prescribing practices","volume":"14","author":"M K Wally","year":"2023","journal-title":"Appl Clin Inform"},{"issue":"06","key":"ref44","doi-asserted-by":"crossref","first-page":"1050","DOI":"10.1093\/jamia\/ocac027","article-title":"Clinician collaboration to improve clinical decision support: the Clickbusters initiative","volume":"29","author":"A B McCoy","year":"2022","journal-title":"J Am Med Inform Assoc"}],"container-title":["Applied Clinical Informatics"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/www.thieme-connect.de\/products\/ejournals\/pdf\/10.1055\/a-2595-0317.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,9,12]],"date-time":"2025-09-12T23:55:03Z","timestamp":1757721303000},"score":1,"resource":{"primary":{"URL":"http:\/\/www.thieme-connect.de\/DOI\/DOI?10.1055\/a-2595-0317"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,8]]},"references-count":42,"journal-issue":{"issue":"04","published-online":{"date-parts":[[2025,8,6]]},"published-print":{"date-parts":[[2025,8]]}},"URL":"https:\/\/doi.org\/10.1055\/a-2595-0317","archive":["Portico","CLOCKSS"],"relation":{},"ISSN":["1869-0327"],"issn-type":[{"type":"electronic","value":"1869-0327"}],"subject":[],"published-other":{"date-parts":[[2025,8,20]]},"published":{"date-parts":[[2025,8]]}}}